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1  INTRODUCTION 

1.1  EBO  EndState  Objectives 

Any  effects-based  operations  (EBO)  approach  to  plan,  execute,  and  assess  air 
operations  must  account  for  the  fact  that  opponents  are  intelligent,  devious,  and 
proactive.  The  effects  of  a  candidate  operation  cannot  be  determined  by  a  linear  cause- 
and-effect  analysis  (e.g.,  bombing  a  bridge  causes  a  convoy  to  stop,  which  prevents  fuel 
from  reaching  armored  forces)  that  link  a  target  to  some  desired  effects.  Why?  Because 
the  opponent  can  react  (e.g.,  reroute  the  convoy  over  another  bridge,  or  draw  fuel  from 
another  depot),  anticipate  (e.g.,  covertly  stockpile  fuel  near  the  area  of  operations,  before 
the  war  begins),  and  otherwise  work  around  our  desired  effects.  Effects  do  not  occur 
because  we  desire  them;  effects  occur  because  we  leave  the  opponent  no  option  other 
than  to  permit  them. 

Because  opponents  can  intervene  between  targets  and  effects,  we  must  anticipate 
their  actions.  This  is  a  key  element  of  military  analysis,  and  raises  both  methodological 
and  technical  challenges.  Methodologically,  we  can  anticipate  their  actions  on  the  basis 
of: 

Capabilities  -  what  they  could  do,  given  the  resources  remaining  at  their  disposal; 

Rationality  -  what  they  should  do,  if  they  were  to  intelligently  pursue  their 
objectives; 

Intentions  -  what  they  would  do,  given  past  practices,  habits,  and  procedures; 

and  get  different  effects  predictions  in  each  case.  Which  one  is  right?  The  correct 
answer  is  "we  cannot  be  sure".  History  is  full  of  examples  of  irrational  action,  concealed 
intentions,  and  underutilized  capabilities.  However,  we  can  anticipate  the  range  of 
opponents'  actions,  factor  them  into  our  effects-based  analyses,  and  apply  sensors  to 
determine  which  of  the  potential  actions  actually  unfold. 

Anticipating  enemy  workarounds  requires  an  understanding  of  the  enemy's 
Centers  of  Gravity  (COGs).  Centers  of  Gravity  are  those  characteristics,  capabilities,  or 
localities  from  which  an  enemy  derives  its  freedom  of  action  (to  work  around  our 
interventions),  its  physical  strength  or  its  will  to  fight.  Opponents'  COGs  take  many 
forms,  from  highly  regularized  engineering  structures  (military  logistics,  infrastructure 
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systems)  to  amorphous  social  structures  (political  support,  leadership  relations). 

Likewise,  the  models  and  technology  required  to  predict  opponents'  actions  take  many 
forms,  ranging  from  analytic  solutions  of  games  over  networks  to  subjective  analyses  of 
social  linkages. 

The  goal  of  EBO  EndState  is  to  develop  and  demonstrate  technology  to  identify 
targets  that  will  generate  desired  effects  and  further  to  assess  the  level  of  the  effects 
brought  about  by  a  targeting  action.  To  perform  these  tasks  the  EBO  Endstate  requires 
access  to  detailed  information  on  enemy  facilities  (location,  function,  capacity,  etc.)  and 
linkages  between  these  facilities  (supply  rate,  commodity,  etc.).  This  Order-of-Battle 
information  is  typically  stored  in  the  Modernized  Integrated  Database  (MIDB).  As  a 
result,  a  critical  feature  of  EBO  EndState  involved  developing  an  interface  to  absorb  and 
understand  information  from  MIDB  databases.  In  addition  to  Enemy  Order-of-Battle 
information,  EndState  functionality  requires  access  to  Effects  based  plans  and  COG 
models.  To  facilitate  these  requirements,  ALPHATECH  (with  AFRL’s  encouragement) 
decided  to  implement  EBO  EndState  as  a  component  of  AFRL’s  Strategy  Development 
Tool  (SDT).  This  provides  EBO  EndState  with  access  to  both  SDT’s  plan  authoring 
information  as  well  as  models  developed  under  SDT’s  COG  Articulator  component.  The 
complete  integration  of  these  components  provides  an  end-to-end  set  of  tools  for  effects 
based  planning,  targeting  and  assessment.  Further,  this  modular  design  allows  EBO 
EndState  to  export  targeting  selections  directly  into  the  effects-based  plan  at  the 
appropriate  point  in  the  plan  hierarchy. 

1.2  EBO  EndState  Capabilities  Summary 

EBO  EndState  capabilities  incorporated  into  the  SDT  product  include  both 
automated  and  manual  targeting  and  target  system  analysis  functions.  The  EndState 
Target  Query  Tool  provides  a  graphic  interface  to  identify  targets  by  Type,  Location, 
Links  (to  other  targets),  Name  or  Target  System.  Results  from  these  queries  are  provided 
in  tabular  form  as  well  as  map  based  displays.  Once  identified,  target  can  be  selected  as 
direct  or  indirect  targets  and  exported  to  Strategy  Development  Tool  (SDT)  based  plans. 
In  addition  to  specifying  targets  for  prosecution,  targets  can  be  designated  with  exclusion 
constraints  (Do  Not  Strike  or  Do  Not  Effect)  and  similarly  exported  to  SDT.  EndState 
functionality  examines  all  prosecuted  targets  to  insure  compliance  of  these  exclusion 
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constraints  (warnings  are  provided  for  constraint  violations).  EndState’s  Option 
Generation  capabilities  use  node  and  link  (when  available)  attributes  to  propose  targets 
that  achieve  desired  effects  as  defined  in  an  SDT  based  plan.  Finally,  EndState’s  Target 
Systems  Analysis  (TSA)  capability  provides  a  qualitative  estimate  of  the  effects  of  a  plan 
on  interrelated  networks.  These  estimates  are  provided  to  the  warfighter  as  graphs  of 
production  and  consumption  capabilities  over  time,  again  this  information  is  fully 
integrated  with  the  SDT  plan  authoring  system  and  the  map  based  display.  Further 
details  of  these  capabilities  and  the  technology  behind  their  development  are  provided  in 
subsequent  sections. 

1.3  Document  Organization 

This  final  report  for  the  EBO  EndState  program  is  organized  as  follows: 

•  Section  2  provides  technical  accomplishments  of  EBO  EndState 
development  performed  for  AFRL 

•  Section  3  presents  software  requirements  for  the  AFRL  EBO  EndState 
software  development 

•  Section  4  presents  the  AFRL  EBO  EndState  System  Specification 

•  Section  5  provides  software  design  for  the  AFRL  EndState 
implementation 

•  Section  6  provides  highlights  and  lessons  learned  from  JEFX  04 

•  Section  7  Provides  a  summary  of  work  accomplished  for  the  DARPA 
EndState  effort 

•  Appendixes  A-F  provide  background  material  and  references 

2  AFRL  ENDSTATE  TECHNICAL  ACCOMPLISHMENTS 
2.1  Target  System  Analysis  (TSA) 

EBO  Endstate  has  implemented  a  framework  for  cross-network  target  system 
analysis.  This  framework  allows  the  integration  of  simulation  models  that  work  on 
different  timescales  or  even  different  level  of  data  aggregation.  It  is  based  on  a 
micro-economic  model  developed  by  a  consultant  [Dahleh],  The  framework 
integrates  models  that  do  not  need  to  be  aware  of  each  other,  to  simulate  the  effects  of 
network  damage  as  the  effects  propagate  through  the  networks  without  requiring  a 
tight  coupling  between  the  models. 
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The  interface  is  done  through  a  simple  XML-based  protocol,  which  allows  models  to 
be  added  and  removed  with  ease.  Early  in  the  contract,  we  developed  complicated, 
physics-based  models  for  the  initial  use,  which  ran  as  external  programs.  Later  in  the 
contract,  these  models  were  replaced  with  models  based  on  the  data  available  from 
the  MIDB.  No  modifications  to  the  central  algorithm  were  required  for  this  change. 

When  provided  with  a  plan,  a  list  of  direct  and  indirect  targets,  and  no-strike  and 
no-effect  lists,  the  TSA  module  provides  a  description  of  the  effects  that  targeting  the 
direct  targets  will  have  on  the  indirect  target.  For  each  target  (direct  or  indirect)  as 
well  as  nodes  on  the  no-strike  and  no-effect  list,  the  module  generates  a  time  profile 
indicating  the  level  of  activity  at  the  node. 

Central  to  the  module  is  a  micro-economic  algorithm.  A  set  of  prices  is  created 
for  all  commodities.  These  prices  are  internal  to  the  model  and  not  visible  to  the  user. 
Real-world  price  data  is  not  required.  The  price  of  each  commodity  is  location 
dependent  and  piecewise  constant  in  time.  Each  model  is  given  these  prices  and  then 
performs  an  internal  optimization  of  how  to  best  control  its  network  with  the  current 
prices.  The  values  and  locations  of  externally  demanded  commodities  are  provided  to 
a  central  coordinator,  along  with  data  about  the  location  of  commodities  supplied  for 
other  networks.  The  central  coordinator  then  examines  the  balance  of  supply  and 
demand  at  each  location  and  in  each  time  interval.  If  supply  exceeds  demand  at  a 
given  location,  the  price  is  lowered  based  on  the  amount  of  the  excess  and  the 
duration  in  time.  Likewise,  if  the  demand  exceeds  the  supply,  the  price  is  raised. 

The  algorithm  understands  the  concept  of  inventory.  If  supply  shortfall  is  less 
than  the  current  inventory,  the  prices  are  not  adjusted  but  the  inventory  level  is 
lowered.  Similarly,  an  excess  of  supply  does  not  modify  the  prices  if  there  is 
sufficient  inventory  capacity  to  hold  the  excess  commodities. 

After  all  of  the  prices  are  modified,  they  are  returned  to  the  models  and  the 
process  is  repeated  until  supply  and  demand  have  converged.  An  adaptive  algorithm 
controls  the  size  of  the  change  in  price  to  speed  convergence  and  prevent  repeated 
oscillations  in  the  prices. 
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The  breakpoints  in  the  price  values  are  dynamically  generated.  They  naturally 
occur  at  times  when  targets  are  struck,  but  they  can  also  be  generated  through  an 
imbalance  in  supply  and  demand  combined  with  a  limited  inventory.  If  the  inventory 
is  exhausted  (or  filled),  the  price  will  rise  (or  fall)  immediately  afterwards, 
representing  that  the  demand  is  no  longer  being  met. 

Several  types  of  models  were  developed  to  support  the  TSA  analysis.  The  most 
straightforward  are  the  infrastructure  models.  These  are  based  on  the  physical 
infrastructure.  They  use  a  linear  program  to  model  the  flow  within  a  single  network. 
Piecewise  linear  costs  are  used  in  the  objective  function  to  provide  stability  when  the 
costs  change  and  to  achieve  a  realistic  allocation  when  nodes  are  consuming  at  less 
than  full  capacity. 

The  second  type  of  model  is  an  end-user  model.  Civilian  and  military  users 
consume  the  commodities,  but  do  not  produce  anything  that  is  covered  within  the 
TSA  analysis.  To  handle  this,  a  “constant  demand”  model  was  developed.  The 
model  normally  returns  a  constant  value  for  the  demand,  unless  the  price  for  the 
commodity  rises  above  a  threshold.  In  that  case,  the  demand  begins  to  decrease 
slowly,  until  it  eventually  reaches  zero. 

The  last  model  was  the  leadership  model.  The  market-based  economics  algorithm 
that  is  central  to  the  TSA  module  will  reach  the  most  “efficient”  solution,  as  would  be 
expected  in  a  country  with  a  free  market.  The  countries  against  which  we  typically 
would  apply  TSA  are  generally  autocratic  countries  with  a  state  directed  economy. 

As  such,  even  if  it  is  more  efficient  to  spread  a  shortage  over  everyone,  the  leadership 
is  likely  to  insist  (for  example)  that  the  military  receive  full  supplies,  even  if  civilians 
receive  very  little  or  nothing.  To  model  this,  we  created  a  Bayes  net  to  model  the 
probability  of  the  leadership  achieving  its  goals  based  on  (among  other  factors)  the 
supply  of  various  commodities  to  different  sectors.  When  a  particular  sector  is  being 
shorted,  it  increases  the  demand  of  an  aggregate  commodity  representing  the  supply 
to  that  sector.  This  behaves  similarly  to  a  subsidy  for  that  sector,  so  that  supply  will 
increase.  The  model  performs  an  optimization,  balancing  the  probability  of  achieving 
its  objectives  against  the  cost  of  the  subsidies.  In  our  models,  this  typically  results  in 
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the  military  receiving  full  supply  whenever  the  network  actually  supports  such  a 
situation. 

2.2  Option  Generation  (OG) 

EBO  Endstate  has  implemented  a  framework  for  generating  targeting  options. 

The  framework  reads  facility  data  from  the  MIDB  in  addition  to  using  plan  data  from 
SDT.  If  the  MIDB  contains  link  data,  that  data  is  also  used,  but  link  data  is  optional.  The 
Option  Generator  requires  a  list  of  indirect  targets  from  SDT  as  well  as  a  no-strike  list 
and  a  no-effect  list.  It  applies  several  algorithms  depending  on  the  data  available  in  the 
MIDB  and  information  from  the  plan. 

Common  components 

Regardless  of  the  data  available,  the  framework  begins  with  some  common  steps. 
It  searches  the  plan  for  effects  with  indirect  targets.  It  then  looks  at  the  leaf  effect  or  task 
of  those  effects,  and  determines  whether  we  are  seeking  to  affect  production  of  resources, 
transportation  of  resources  or  both.  (In  all  cases  where  it  is  unable  to  determine  intent 
from  the  plan,  it  assumes  all  possible  options  are  desired.)  It  also  looks  at  the  leaf 
element  to  determine  which  network  (e.g.,  EP  or  POL)  the  user  desires  to  affect.  It  then 
queries  the  MIDB  to  find  all  possible  targets  that  match  these  requirements  and  excludes 
those  that  are  on  the  no-strike  list  or  the  no-effect  list.  The  framework  then  evaluates 
each  possible  target  using  the  algorithms  described  below.  Any  target  selected  by  at  least 
one  algorithm  is  sent  to  the  Map  Tool  as  a  proposed  target,  where  the  user  can  view  the 
target  on  the  map  and  see  which  parts  of  the  plan  the  target  supports.  If  the  user  agrees 
with  the  option  generator,  the  target  can  be  added  to  the  plan. 

Node  based  algorithms 

If  only  node  data  is  available,  the  Option  Generator  uses  two  algorithms.  The  first 
algorithm  is  used  if  the  plan  indicates  that  targeting  of  production  facilities  is  desired.  If 
the  plan  indicates  that  targeting  of  transportation  facilities  is  desired,  then  the  second 
algorithm  is  used.  If  the  plan  is  ambiguous,  then  both  algorithms  are  used  and  the  results 
are  combined. 
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In  both  algorithms,  the  affected  targets  are  grouped  into  clusters.  This  allows 
several  small  but  nearby  targets  to  be  treated  as  a  single  larger  target,  which  can  improve 
the  performance  of  the  algorithms.  For  example,  consider  10  targets  that  each  consumes 
one  unit  are  close  to  5  facilities  that  each  produces  2  units.  If  the  affected  targets  are 
considered  separately,  the  algorithm  might  select  one  of  the  production  facilities  when 
each  affected  target  is  considered  and  then  terminate,  since  the  production  capacity  is 
notably  larger  than  the  consumption  capacity.  By  grouping  the  affected  targets,  the 
consumption  capacity  is  increased  and  the  algorithm  will  select  more  of  the  production 
facilities. 

The  first  algorithm  is  used  when  the  plan  indicates  that  we  desire  to  target 
production  facilities.  The  basic  concept  is  to  locate  targets  that  are  likely  to  supply  the 
affected  target.  Each  group  of  affected  targets  is  considered  in  turn,  and  proposed  targets 
are  generated  for  the  group.  All  of  the  proposed  targets  are  combined  into  a  single  set  at 
the  end  of  the  algorithm.  The  algorithm  assigns  a  score  to  each  possible  target  based  on 
the  capacity  of  the  target  and  the  distance  from  the  indirect  targets.  Larger  capacities  and 
shorter  distances  give  higher  scores.  It  then  sets  a  threshold  based  on  the  typical  score  of 
possible  targets  near  indirect  targets.  Any  possible  target  whose  score  exceeds  the 
threshold  is  included  in  the  generated  target  list. 

The  second  algorithm  is  based  on  the  observation  that  to  suppress  the  flow  of 
commodities,  targets  need  to  be  selected  from  all  possible  directions  of  approach. 
Otherwise,  the  flow  from  an  uncovered  direction  will  increase  to  compensate  for  the 
decrease  from  the  other  directions.  The  algorithm  must  also  be  able  to  handle  the  case 
where  there  are  no  targets  in  a  given  direction,  such  as  when  the  affected  target  is  on  the 
coast.  This  algorithm  adopts  concepts  from  Coulomb’s  law  in  electrostatic  physics.  As 
with  the  first  algorithm,  each  group  of  affected  targets  is  considered  and  the  proposed 
targets  from  each  group  are  combined  to  form  the  final  list  of  proposed  targets.  Each 
indirect  target  is  assigned  a  positive  charge  and  each  possible  target  is  assigned  a  negative 
charge,  with  the  charges  being  proportional  to  the  capacity.  The  possible  target  with  the 
largest  attractive  force  towards  the  affected  target  is  selected.  The  force  calculations  are 
redone,  but  the  repulsive  force  of  the  selected  targets  is  also  included.  This  encourages 
the  selection  of  targets  that  are  one  the  opposite  side  of  the  indirect  target,  as  the 
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previously  selected  proposed  target  “screens”  the  affected  target.  The  process  repeats 
until  no  possible  target  has  an  attractive  force. 

Link  Based  Algorithms 

When  link  data  is  also  available,  the  Option  Generator  uses  three  algorithms.  One 
algorithm  is  used  when  the  plan  indicates  that  production  facilities  are  to  be  targeted, 
while  the  other  two  are  always  used.  The  resulting  target  lists  are  merged. 

With  these  algorithms,  the  affected  targets  are  not  grouped  together.  Instead,  the 
algorithms  consider  all  of  the  affected  targets  at  once,  so  that  a  global  analysis  is 
performed.  Groups  of  smaller  affected  targets  are  not  neglected  in  favor  of  less 
numerous  but  larger  affected  targets,  as  the  global  analysis  will  recognize  the  effect  on 
several  smaller  targets  in  the  same  manner  as  an  effect  on  a  single  target  with  the  same 
aggregate  capacity. 

The  first  algorithm  is  based  on  a  shortest  path  algorithm.  The  concept  here  is  that 
nearby  producers  are  the  likely  suppliers,  so  targeting  them  will  be  helpful.  It  begins  by 
creating  a  graph  representing  the  network  structure.  The  graph  includes  cross-network 
arcs,  so  the  proposed  targets  may  be  in  any  network  that  can  supply  (directly  or 
indirectly)  the  indirect  targets.  It  then  applies  a  shortest  path  algorithm  to  find  the 
geographically  nearest  producers.  It  adds  these  facilities  to  the  proposed  target  list,  until 
the  sum  of  their  capacity  exceeds  the  capacity  of  the  indirect  target  multiplied  by  a  scale 
factor. 

The  second  algorithm  is  based  on  disrupting  transportation,  so  that  flow  from  the 
producers  to  the  consumers  is  not  possible.  This  creates  the  possibility  of  finding  a  small 
set  of  possibly  distant  targets  that  still  achieve  the  desired  effect.  It  also  starts  by  creating 
a  graph  of  the  network.  It  applies  a  modification  of  the  maximum  flow-minimum  cut 
(MFMC)  algorithm  to  find  the  minimal  set  of  nodes  that  will  separate  the  network,  so  that 
no  indirect  target  is  connected  to  any  producer.  The  facilities  corresponding  to  the 
minimal  set  is  returned  as  the  proposed  target  list.  Several  modifications  of  the  basic 
MFMC  algorithm  were  needed  for  this.  The  standard  MFMC  algorithm  assumes  that  any 
link  can  be  selected,  while  we  only  want  to  select  links  that  have  a  possible  target  on 
them.  To  handle  this,  we  increase  the  capacity  of  links  that  are  not  connected  to  targets 
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so  that  they  will  not  be  selected.  We  also  want  a  set  of  facilities,  not  a  set  of  links.  This 
is  done  as  a  post  processing  step.  Once  the  links  are  selected,  we  solve  an  assignment 
problem,  minimizing  the  total  capacity  of  the  selected  targets. 

The  third  algorithm  attempts  to  locate  the  facilities  that  will  have  the  greatest 
effect.  It  generates  a  linear  program  (LP)  to  simulate  the  flows  through  the  networks  and 
the  transformation  of  one  commodity  into  another  at  plants.  It  then  solves  the  LP  by 
maximizing  consumption  at  the  indirect  targets.  The  dual  values  of  the  solution  are 
examined  to  determine  which  nodes  in  the  network  would  have  the  largest  effect,  using 
the  dual  value  multiplied  by  the  capacity  as  an  estimate  of  the  effect.  The  LP  is  modified 
to  represent  the  effect  of  targeting  those  nodes,  and  the  LP  is  resolved.  This  process  is 
iterated  until  the  consumption  at  the  indirect  targets  drops  to  the  value  indicated  in  the 
SDT  plan. 

2.3  Visualization  (Target  Set  Tool  (TST)  and  Query  Tool  (QT)) 

EBO  Endstate  has  implemented  a  visualization  tool  for  examining  and  querying 
physical  networks.  The  tool  is  built  on  top  of  BBN’s  OpenMap  tool 
(http://openmap.bbn.comL  It  provides  several  functions:  display  of  physical  networks, 
manual  targeting,  query  functions  and  display  of  DTED  data.  It  also  provides  graphical 
display  of  results  from  Target  Systems  Analysis  and  Option  Generation  as  described  in 
the  preceding  sections. 

The  tool  displays  node  and  (when  available)  link  data,  overlaid  on  top  of  a 
political  background.  Nodes  can  display  the  name  of  the  node,  and  access  to  more 
detailed  information  is  available  through  use  of  the  mouse. 

The  user  can  manually  select  nodes  and  flag  them  as  either  direct  or  indirect 
targets.  These  nodes  can  than  be  exported  to  SDT  and  added  to  the  plan.  The  capture 
(Figure  1)  below  depicts  this  capability  as  viewed  in  SDT  [Pioch], 
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Figure  1  -  EndState  Target  Set  Tool  (SDT  Capture) 

Nodes  can  also  be  color  coded  according  to  type  as  selected  by  the  user.  The 
configurable  legend  shown  below  depicts  how  specific  node  types  can  be  color  coded  for 
easy  identification  on  the  map  display  (see  Figure  2). 


Figure  2  —  EndState  Map  Legend 

Endstate  visualization  capabilities  include  a  Query  Tool  that  allows  the  user  to  search  for 
nodes  according  to  specific  criteria.  The  user  can  search  for  nodes  by  name,  BE  number, 
category  code,  distance  from  other  nodes,  or  by  target  system.  The  results  can  be 
selected  for  display  on  the  map  and  flagged  as  direct  or  indirect  targets  (graphic  below  - 
Figure  3). 
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Figure  3  —  EndState  Query  Tool 

There  is  also  an  interface  for  displaying  the  results  of  Option  Generation.  This 
allows  the  user  to  approve  the  results  of  the  algorithms  and  add  them  to  the  plan  (see 
Figure  4  below).  In  this  graphic,  we  see  a  number  of  nodes  (facilities  from  the  MIDB) 
selected  by  the  Option  Generation  capability  as  potential  targets  which  support  a  given 
task  to  satisfy  a  particular  effect  defined  in  the  plan.  These  potential  targets  are  presented 
along  with  the  entry  point  (task  node)  in  the  plan  where  they  have  been  identified.  The 
user  may  select  a  set  of  these  potential  targets  (highlighted  in  green)  for  direct  export  to 
the  plan.  This  export  of  targets  to  the  plans  is  executed  by  selecting  the  “bullseye”  button 
in  the  Target  Set  Tool  once  targets  have  been  highlighted. 
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Figure  4  —  EndState  Option  Generation  Results  Display 
In  addition  to  Option  Generation  results,  Endstate  visualization  capabilities  depict  the 
results  of  Target  Systems  Analysis  over  time.  The  captures  below  (Figure  5)  show 
estimated  operational  capability  of  nodes  (as  indicated  by  bars  associated  with  the  node) 
at  two  distinct  points  in  time.  The  Target  Systems  Analysis  (as  described  above) 
propagates  the  impact  of  targeting  actions  over  time.  Hence,  we  see  a  degradation  in 
node  performance  as  the  effects  of  those  targeting  actions  propagate  through  the  system. 
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Figure  5  —  EndState  Target  Systems  Analysis  Results  Display 
Finally,  EBO  EndState  has  implemented  methods  to  specify  targets  as  “Do  Not  Strike”  or 
“Do  Not  Effect”  targets.  This  capability  stores  such  designations  with  target  information 
and  analyzes  each  targeting  decision  based  on  these  conditions.  Attempting  to  strike  a 
target  designated  as  “Do  Not  Strike”  or  attempting  to  effect  targets  designated  as  “Do  Not 
Effect”  will  result  in  an  error  (Figure  6). 
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3  AFRL  ENDSTATE  SOFTWARE  REQUIREMENTS 
3.1  System/Subsystem  Specification 

The  purpose  of  EndState  software  is  to  receive  a  plan  from  SDT  and  send  a  prediction  of 
that  plan’s  outcome  back  to  the  SDT.  The  predicted  outcome  will  be  based  upon 
simulations  in  a  number  of  models  known  as  COGs.  Endstate  software  should  be  flexible 
enough  to  allow  for  additions  of  new  COGs.  Each  COG  will  model  a  specific  aspect  of 
the  plan  such  as  Electric  Power  or  the  road  networks.  Endstate  software  must  fulfill  the 
following  requirements: 

1 .  Communicate  with  SDT 

2.  Mechanism  for  communication  between  Endstate  components 

3.  Communicate  with  COGs 

4.  Coordinate  communication  between  COGs 

5.  Coordinate  time  between  COGs  during  simulation 

6.  Produce  a  prediction  about  the  SDT  plan’s  outcome 

7.  Provide  a  graphical  map  tool 

8.  Allow  queries  of  facilities  in  the  map  tool 

9.  Generate  targeting  options 

10.  Report  on  undesired  effects 

1 1 .  Allow  the  user  to  specify  facilities  that  should  not  be  affected  or  targeted 

12.  Produce  a  report  of  the  effects  at  facilities  that  should  not  be  affected 

13.  Support  queries  related  to  the  target  system  templates  developed  for  JEFX  04 
Endstate  Software  will  be  written  as  much  in  Java  1 .4  as  possible  but  only  guaranteed  to 
run  on  windows  NT.  There  will  be  5  main  components;  the  Coordinator,  Predictor, 
Temporalizer,  Target  SetTool,  and  Option  Generator.  Only  the  Target  Set  Tool  will  have 
a  GUI  interface  viewable  by  the  user.  Software  from  the  SDT  or  COGs  may 
communicate  with  Endstate  via  XML  messages  sent  over  sockets  to  the  Coordinator.  A 
user  or  SDT  software  may  start  the  Endstate  software  via  a  software  call  or  the  user 
choosing  the  exit  button  on  the  Coordinator  may  close  it. 

Coordinator 

The  Coordinator  will  act  as  an  intelligent  server.  It  will  accept  inputs  of  XML  messages 
and  output  XML  messages  to  the  appropriate  software  component  using  a 
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publish/subscribe  method.  Software  components  that  communicate  with  the  Coordinator 
are  the  Predictor,  Temporalizer,  Option  Generator,  Target  Set  Tool  and  the  SDT 
Software.  The  Coordinator  will  also  know  about  all  of  the  COGs  as  they  are  added  as 
well  as  how  to  coordinate  communication  with  each  COG.  There  will  be  a  GUI  with  a 
window  displaying  the  current  communication  messages  as  well  as  an  exit  button  to  close 
Endstate  software.  This  window  will  only  be  visible  when  the  software  is  configured  for 
development. 

The  predictor  takes  as  inputs,  via  messages  from  the  Coordinator,  the  desired  actions 
from  the  SDT,  the  target  sets  from  the  SDT,  and  predictions  from  the  COGs.  The 
predictions  from  the  COGs  at  each  time  step  during  each  COG’s  simulation  are  matched 
to  the  desired  actions  and  targets  to  create  graphs  charting  the  predicted  probability  of 
each  target  and  action  over  time.  The  resulting  graphs  are  sent  to  the  Coordinator  and 
received  by  the  SDT. 

The  Temporalizer’ s  function  is  to  coordinate  simulation  time  information  between  COGs. 
The  Temporalizer  receives  as  inputs  a  list  of  COGs  that  are  used  in  Endstate  to  obtain 
predictions  about  a  plan  and  coordinates  time  between  those  COGs  as  they  simulate. 

After  all  COGs  have  finished  their  simulations,  the  Temporalizer  sends  a  message  to  alert 
that  all  simulations  are  complete. 

The  SDT  provides  the  target  set  for  simulations.  The  user  inputs  targets  via  a  GUI  and 
targets  are  sent  to  COGs  as  well  as  the  Predictor  for  initializing  simulations.  The  targets 
set  by  the  SDT  are  targets  that  will  be  used  to  facilitate  the  plan  obtained  from  the  SDT. 

Option  Generation 

This  section  covers  item  7  above. 

The  Option  Generator  (OG)  will  work  with  data  available  from  the  SDT  plan  and 
from  the  secret  level  MIDB.  As  such,  the  OG  will  work  with  the  SDT  in  theater.  The 
OG  will  perform  several  tasks: 

1 .  Query  the  MIDB  for  data  on  facilities  and  their  attributes 

2.  Employ  heuristic  algorithms  to  generate  candidate  target  sets 

3.  Automatically  generate  target  sets  from  the  Effects-Based  Plan 
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4.  Include  visualizations  using  the  Target  Set  Tool 

5.  Allow  updates  from  intelligence  and  BDA  to  refine  target  sets 

6.  Be  fully  integrated  with  the  SDT  and  existing  Endstate  components 
The  following  subsections  provide  details  of  these  requirements. 

Query  MIDB 

The  OG  will  interface  with  the  MIDB  using  SQL  queries.  This  will  provide 
information  about  nodes  but  not  about  links.  The  initial  MIDB  will  be  the 
unclassified  MIDB  mockup  provided  by  Air  Force  Research  Laboratory  -  Rome. 

That  database  must  have  the  same  schema  as  the  current  secret  MIDB.  They  fields 
should  also  be  populated  in  the  same  manner  as  the  secret  MIDB.  If  these 
requirements  are  met,  transitioning  to  using  the  real  MIDB  should  be  simple  and 
straightforward. 

Heuristic  algorithms 

The  heuristic  algorithms  used  by  the  OG  will  generate  a  target  set  based  on  the 
information  in  the  SDT  plan,  the  node  information  in  the  MIDB  and 
BD A/intelligence  data  in  the  MIDB.  Details  of  the  algorithms  will  be  determined 
during  the  analysis/design  phase.  It  is  likely  that  several  iterations  will  be  required  to 
finalize  the  algorithms. 

Generate  target  sets 

The  OG  will  run  the  heuristic  algorithms  to  generate  a  target  set.  During  this  phase, 
the  output  will  be  deterministic.  Given  an  SDT  plan,  an  MIDB  database  and  a 
specific  set  of  BDA  reports,  the  OG  will  generate  exactly  one  target  set.  If  queried 
again,  it  will  generate  the  same  target  set. 

The  OG  will  subscribe  to  plan  messages  from  the  XML  server.  Upon  receiving  a 
plan  message  containing  affected  targets,  it  will  respond  with  a  proposedTargetSet 
message.  The  format  of  the  proposedTargetSet  message  will  contain  elements  similar 
to  those  in  the  targetSet  and  augmentation  messages.  The  portions  similar  to  the 
targetSet  message  will  provide  a  list  of  the  proposed  targets.  The  parts  similar  to  the 


16 


augmentation  message  will  provide  a  skeleton  plan  fragment,  to  link  the  targets  with 
the  lowest  level  nodes  in  the  existing  plan.  The  details  will  be  finalized  during  the 
design  process. 

Visualization 

The  current  Target  Set  Tool  (TST)  will  be  extended  to  show  the  generated  target  set. 
It  will  subscribe  to  the  proposedTargetSet  message  and  update  its  display  when  the 
message  is  received. 

The  Strategy  Development  Tool  (SDT)  must  add  the  appropriate  nodes  to  the  plan 
when  the  user  approves  the  proposed  target  set. 

Refining  target  sets 

If  the  intelligence  or  BDA  information  in  the  MIDB  is  changed,  the  OG  may  generate 
a  different  target  set.  This  will  be  a  manual  process,  requiring  the  user  to  request  an 
updated  target  set. 

Integration  with  the  SDT  and  existing  Endstate  components 

The  OG  will  be  fully  integrated  with  the  SDT  and  the  TST.  It  is  philosophically 
inconsistent  with  the  Cross  Model  Coordinator  (XMC)  framework.  No  effort  will  be 
made  to  integrate  the  OG  and  the  XMC  or  to  avoid  conflicts  if  both  are  used 
simultaneously. 

Networks 

The  Phase  I  option  generator  for  facilities  will  support  the  following  networks: 
Electric  Power 
T  elecommunications 

Petroleum,  Oil  and  Lubricants  and  Natural  Gas 
Rail  and  Road  LOC 

Do  Not  Effect  Facilities 

This  section  covers  items  10,  11,  and  12. 
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The  intent  of  this  enhancement  is  to  allow  the  user  of  SDT  to  flag  facilities  that 
should  not  be  directly  targeted  (Do  Not  Target  (DNT))  or  indirectly  affected  (Do  Not 
Effect  (DNE)).  The  user  will  receive  feedback  if  attempts  to  violate  these  constraints 
occur. 

The  following  requirements  must  be  met. 

1 .  The  user  must  be  able  to  select  an  individual  facility  as  a  DNE  or  DNT. 

2.  The  user  must  be  able  to  select  a  group  of  facilities  by  category  code  as 
DNE  or  DNT. 

3.  If  the  user  attempts  to  select  a  facility  on  the  DNT  list  as  a  direct  target,  he 
will  be  notified  and  prevented  from  doing  so. 

4.  If  the  user  attempts  to  select  a  facility  on  the  DNE  list  as  a  direct  or  an 
indirect  target,  he  will  be  notified  and  prevented  from  doing  so. 

5.  Outage  profiles  will  be  provided  for  DNE  targets.  These  will  be  similar  to 
what  is  currently  provided  for  indirect  and  direct  targets.  This  will  only 
happen  if  the  effect  node  contains  a  description  that  Endstate  understands, 
similar  to  the  current  restriction  on  indirect  target  and  outage  profiles. 

6.  If  an  outage  profile  is  provided,  and  the  facility  is  affected  beyond  the 
limits  stated  in  the  effect,  the  user  will  be  notified. 

7.  Both  the  DNE  and  DNT  lists  will  be  global.  A  future  version  may  provide 
DNE/DNT  lists  that  are  limited  to  a  specific  phase,  effect,  etc. 

TSA  Queries 

This  section  covers  item  13  above. 

The  C2TIG  has  identified  seven  target  systems  that  are  to  be  involved  in  the  JEFX 
exercise.  These  systems  are  as  follows: 

1 .  National  Command  and  Control 

2.  C4I  and  Telecommunications 

3.  Air  Defense  (IADS) 
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4.  Fielded  Forces 


5.  Electrical  Power 

6.  POL 

To  support  work  in  with  these  systems,  the  software  will  allow  the  user  to  specify  a 
target  system  and  restrict  the  Query  Tool  selection  to  just  facilities  that  belong  to  that 
system. 

Requirements 

The  following  requirements  will  implement  this  task. 

1 .  The  user  must  be  able  to  select  facilities  for  each  target  system.  This  requires  an 
additional  panel  in  the  Query  Tool. 

2.  The  software  must  convert  the  target  system  into  a  collection  of  category  codes, 
according  to  the  table  below.  In  the  category  codes,  x  is  used  where  any  digit  is 
acceptable.  Category  code  definitions  can  be  found  in  Ref.  MIDB. 
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Target  System 

Category  Codes 

National  Command  and 

Control 

41xxx,  527xx,  741xx,  7741x,  759xx,  773xx,  775xx,  82xxx, 

874xx,  89xxx,  91xxx,  93xxx,  941xx,  96xxx 

C4I  and 

Telecommunications 

41xxx,  74xxx,  774xx,  82xxx,  83xxx,  899xx,  93xxx,  94xxx, 

96xxx,  981 xx 

Air  Defense  (IADS) 

80xxx,  81xxx,  82xxx,  85xxx,  86xxx,  872xx,  873xx,  877xx, 

879x4,  879x5,  882xx  ,  89003,  983xx 

Fielded  Forces 

63xxx,  642xx,  646xx,  649xx,  662xx,  663xx,  664xx,  666xx, 

668xx,  669xx,  691xx,  67xxx,  752xx,  76101,  7631x,  7813x, 

7814x,  80xxx,  81xxx,  84xxx,  85xxx,  86xxx,  87xxx,  88xxx, 

89xxx,  90xxx,  91 xxx,  92xxx,  941xxx,  95xxx,  96xxx,  97xxx, 

98xxx 

Electrical  Power  (EP) 

32xxx,  42xxx 

Petroleum,  Oil  and 

Lubricants  (POL) 

14xxx,  15xxx,  210xx,  211xx,  212xx,  213xx,  214xx,  215xx, 

217xx,  218xx,  372xx,  4072x 

The  category  codes  for  EP  and  POL  are  taken  from  the  MIDB  SOP. 
Command,  Control,  Computers  and  Intelligence  (C4I) 


The  category  codes  for  C4I  and  Telecommunications  are  based  on  the  Califon 
Command,  Control,  Communications,  Computers,  and  Intelligence  (C4I)  Target  System 
Analysis.  It  does  not  give  explicit  category  codes,  but  instead  provides  guidelines  for 
requested  data.  From  these  requests,  we  extrapolated  the  following  category  codes.  The 
software  will  not  subdivide  the  system  according  to  the  bullets  below,  so  the  exact 
association  of  category  codes  with  the  individual  items  is  not  important. 
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Governmental  facilities  -  74xxx,  774xx,  93xxx,  94xxx 
Telecommunication  links  and  nodes  -  41xxx 
TV,  radio,  wireless  phone,  SATCOM 
Broadcast  stations,  repeater  stations,  microwave  towers 
Network  hubs,  fiber  optics  and  coaxial  cable 
Computer  network  facilities 

Facilities  with  Information  Operations  or  PSYOP  capability  -  83xxx 
National  intelligence  facilities  -  899xx 
SIGINT  or  HF/DF  sites 

SATCOM  intercept,  ELINT  and  intelligence  related  communications 
Intelligence  HQ 

Air  Operation  Centers  (ADOC,  ZOC,  IOC,  RCC)  -  82xxx 
Army  Corps  and  Naval  Flotilla  and  Costal  Defense  HQ  -  96xxx,  981xx 
Special  Operation  Forces  facilities 
Fielded  Forces 

The  category  codes  for  Fielded  Forces  are  based  on  the  Califon  Fielded  Forces 
Target  System  Analysis  (Ref.  TSA-AFRL).  It  does  not  give  explicit  category  codes,  but 
instead  provides  guidelines  for  requested  data.  From  these  requests,  we  extrapolated  the 
following  category  codes.  Category  codes  pertaining  to  production  of  military  supplies 
have  been  excluded,  but  facilities  for  repair/rebuild  have  been  included.  The  software 
will  not  subdivide  the  system  according  to  the  bullets  below,  so  the  exact  association  of 
category  codes  with  the  individual  items  is  not  important. 

Corps  HQ,  garrisons,  depots,  FOL  -  63xxx,  642xx,  646xx,  649xx,  662xx,  663xx, 
664xx,  666xx,  668xx,  669xx,  691xx,  752xx,  76101,  7631x,  7813x,  7814x, 
89xxx,  90xxx,  91 xxx,  92xxx,  941xxx 

Division  HQ 
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Brigade  HQ 

Flotilla  HQ  -  67xxx,  95xxx,  96xxx,  97xxx,  98xxx 

Air  force,  bases,  airfields  -  80xxx,  81xxx,  84xxx,  85xxx,  86xxx,  87xxx,  88xxx 
Special  Operations 
Strategic  Air  Defense 

The  category  codes  for  Air  Defense  are  based  on  the  Califon  Strategic  Air 
Defense  Target  System  Analysis  (Ref.  TSA-AFRL).  It  does  not  give  explicit  category 
codes,  but  instead  provides  guidelines  for  requested  data.  From  these  requests,  we 
extrapolated  the  following  category  codes.  The  software  will  not  subdivide  the  system 
according  to  the  bullets  below,  so  the  exact  association  of  category  codes  with  the 
individual  items  is  not  important. 

Zonal  Operation  Centers 

Regional  Control  Centers 

C3  Battle  Management  -  82xxx,  81xxx,  89003 

SAM  facilities  -  872xx,  879x4,  879x5 

AAA  AD  units  -  873xx,  877xx,  983xx 

Forward  Operation  Locations  -  882xx 

Repair/Supply  Depots  -  86xxx 

Airfields  -  80xxx 

EW/GCI  C2,  Fighter  Direction  Posts,  Reporting  Posts 
Airborne  Early  Warning 
IADS,  IADS/IO  and  IADS  support  -  85xxx 
National  Command  and  Control  (C2) 

The  category  codes  for  National  Command  and  Control  are  based  on  the  Califon 
National  Command  and  Control  (C2)  Target  System  Analysis  (Ref.  TSA-AFRL).  It  does 
not  give  explicit  category  codes,  but  instead  provides  guidelines  for  requested  data.  From 
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these  requests,  we  extrapolated  the  following  category  codes.  The  software  will  not 
subdivide  the  system  according  to  the  bullets  below,  so  the  exact  association  of  category 
codes  with  the  individual  items  is  not  important. 


Governmental  facilities  -  741xx,  7741x 

Military  facilities  and  HQ  -  82xxx,  874xx,  91xxx,  941xx,  96xxx 
Internal  security  facilities  -  89xxx 
Prison/detention  facilities  -  759xx,  775xx 
Police  barracks  -  773 xx 

Communication  nodes,  TV,  radio,  wireless  phone,  SATCOM  -  41xxx,  93xxx 

Broadcast  stations,  repeater  stations,  fiber  optics  and  coaxial  cable,  microwave 
towers,  network  hubs  -  527xx 
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4  SYSTEM/SUBSYSTEM  DESIGN  DESCRIPTION 
4.1  System  Architectural  Design 

Figure  7  depicts  the  basic  components  of  an  “End  to  End”  EBO  thread.  The  shaded 
portions  are  those  developed  under  the  scope  of  this  project.  The  Plan  Editor  was 
developed  by  the  EBO  Strategy  Development  project  while  the  User  and  Cog  Expert 
nodes  represent  Human  users.  Square  nodes  are  used  to  represent  software  modules  and 
rounded  squares  represent  files  or  other  methods  of  data  storage/transfer. 


Figure  7  -  Components  for  EBO  Interactions. 

The  Coordinator  is  the  external  interface  for  the  EBO  Endstate  software.  It 
receives  information  requests  from  the  EBO  SDT  software.  It  obtains  detailed  target  sets 
from  the  SDT  tool.  It  runs  the  COG  models  and  translates  the  output  of  the  COG  tools 
into  a  common  format.  Information  to  the  Temporalizer  or  COGs  is  forwarded  via  the 
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Coordinator.  The  Predictor  will  provide  the  Coordinator  with  probabilities  of  achieving 
the  desired  effects. 

The  Coordinator  is  complex.  Figure  8  shows  the  internal  components  of  the 
coordinator.  It  consists  of  the  following  components. 

•  The  SDT  tool  is  developed  by  the  EBO  SDT  team.  It  sends  a  plan  containing 
the  desired  effects  and  a  high  level  description  of  the  targeting  that  the  plan 
anticipates.  It  provides  the  simulated  effects  and  explanations  to  the  SDT 
tool. 

•  The  Target  Set  Interface  provides  the  target  sets  for  the  COG  tools.  It  is  also 
part  of  the  SDT  tool. 

•  The  Cross-Coordinator  produces  the  simulated  results  of  a  target  set.  It 
coordinates  between  the  COG  models  to  ensure  that  the  results  of  the 
simulations  are  consistent  at  the  global  level.  This  will  be  in  Java. 

•  The  Single  COG  Interfaces  translate  the  target  sets  from  the  common  format 
used  in  the  other  components  into  the  actions  that  are  appropriate  for  a 
specific  COG.  They  translate  the  COG-specific  output  into  a  common  format. 
This  may  be  in  Java  or  C++. 
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Figure  8  -  Coordinator  components.  External  modules  are  shown  in  yellow. 

The  option  generator  is  an  independent  part  of  the  Endstate  software.  (Some  low 
level  classes  will  be  shared.)  It  receives  requests  of  proposed  target  sets  from  the  SDT 
tool.  Based  on  the  data  in  the  request  and  the  type  of  data  available  from  the  MIDB  (e.g., 
does  it  contain  link  data),  the  option  generator  selects  a  group  of  heuristics  to  propose 
targets.  These  proposed  targets  are  sent  to  the  Target  Set  Tool. 

The  target  set  tool  displays  a  map  of  the  region.  On  the  map,  direct,  indirect,  do 
not  affect  and  do  not  strike  facilities  are  indicated.  After  a  simulation,  the  target  set  tool 
can  also  display  the  results  of  the  simulation.  When  the  option  generator  is  invoked,  it 
displays  the  proposed  target  sets  and  allows  the  user  to  export  the  targets  to  the  SDT  plan. 

4.2  Requirements  Traceability 

This  section  provides  a  map  between  the  requirement  and  the  components  defined  in 
this  section. 
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Requirement 

Component 

1  Communicate  with  SDT 

Coordinator 

2  Mechanism  for  communication  between  EndState 

components 

Temporalizer 

3  Communicate  with  COGs 

Predictor 

4  Communicate  communication  between  COGs 

XML  Router 

5  Coordinate  time  between  COGs  during  simulation 

Cross  COG  Coordinator 

6  Produce  a  prediction  about  the  SDT  plan’s  outcome 

COG  Interface 

7  Provide  a  graphical  map  tool 

Target  Set  Tool 

8  Allow  queries  of  facilities  in  the  map  tool 

Target  Set  Tool 

9  Generate  targeting  options 

Option  Generator 

10  Report  on  undesired  effects 

Predictor 

1 1  Specify  facilities  to  not  affect  or  target 

Target  Set  Tool 

12  Produce  a  report  of  the  undesired  effects 

Predictor 

13  Support  queries  for  JEFX 

Target  Set  Tool 

4.3  Software  Requirements  Specifications 


CSCI  Requirements 

There  are  five  major  components:  the  Coordinator,  the  Temporalizer,  the  Predictor, 
Target  Set  Tool  and  the  Option  Generator.  We  also  specify  the  components  that  form 
the  Coordinator. 

Coordinator 

The  Coordinator  consists  of  four  components,  plus  an  additional  component  for  each 
COG.  The  four  components  are  the  SDT  interface,  the  Predictor,  the  cross- 
Coordinator  and  the  Temporalizer  interface. 
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SDT  Tool 


The  Plan  Editor  is  being  created  by  the  EBO  Strategy  Development  Team  (SDT). 
We  therefore  must  work  with  the  SDT  team  in  determining  the  best  approach  for 
communication  between  our  components  and  theirs.  The  following  interface 
specifications  must  be  communicated  between  EBO  EndState  and  SDT. 

1 .  Coordinator  requires  desired  effects  and  targeting  guidance,  which  the 
Plan  Editor  will  provide. 

2.  The  Plan  Editor  requires  the  target  set,  simulated  effects  and  an 
augmentation  for  the  Bayesian  net,  which  provides  detail  relating  to  the 
current  plan.  The  Coordinator  will  provide  this. 

3.  File  format  for  data  saved  by  Plan  Editor 

4.  File  format  for  data  saved  by  Coordinator 

5.  A  way  for  Coordinator  to  know  that  the  Plan  Editor  has  finished  saving 
data  and  Coordinator  should  now  run  the  appropriate  model  file 

6.  A  way  for  Plan  Editor  to  know  that  Coordinator  has  finished  running  and 
it  should  gather  results  to  show. 

Cross-Coordinator 

This  module  runs  the  side-to-side  thread.  As  input,  it  receives  the  target  set  and  the 
desired  effects.  It  returns  an  internally  consist  result  from  running  the  COG 
simulations,  including  the  simulated  effects.  (Need  more  detail  when  S2S  thread  is 
better  defined.)  Additionally,  it  needs  to  be  aware  of  what  types  of  desired  effects 
can  be  addressed  by  an  individual  COG. 

COG  Interface 

The  COG  interface  will  interact  with  the  COG  models,  requesting  the  appropriate 
data  from  the  model  and  forwarding  it  to  the  XML  router. 
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XML  Router 


The  Coordinator  will  reuse  the  XML  router  from  the  Active  Templates  project  for 
routing  XML  messages. 

Target  Set  Tool 

This  GUI  will  be  used  by  a  COG  expert  to  enter  a  detailed  target  set.  It  will  provide 
the  expert  with  a  summary  of  the  plan  to  aid  in  determining  the  target  set. 

Temporalizer 

This  module  controls  time.  It  sends  messages  indicating  the  current  time  in  the 
simulation,  and  receives  replies  from  all  components  that  are  interested  in  time. 
When  it  determines  that  all  components  have  reached  an  endstate,  it  sends  a 
simulation  done  message. 

Predictor 

This  module  propagates  the  results  of  the  simulations  in  a  Bayesian  network  through 
time  and  produces  probabilities  of  achieving  desired  effects  along  with  mechanisms 
to  explain  the  effects.  It  is  written  in  Java.  See  interface  specifications  for  details. 

Option  Generator 

This  module  generates  targeting  options.  It  receives  messages  from  the  SDT 
containing  the  plan.  It  sends  messages  containing  proposed  targets  for  achieving  the 
effects  listed  in  the  plan.  It  uses  heuristics  to  generate  those  targets.  It  is  written  in  Java. 
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5  AFRL  ENDSTATE  SOFTWARE  DESIGN 
5.1  SOFTWARE  ARCHITECTURE 

The  software  consists  of  three  main  components.  These  are 

1 .  Target  Set  Tool  -  provides  a  map-based  GUI 

2.  Coordinator  -  performs  target  system  analysis 

3.  Option  Generator  -  creates  sets  of  proposed  targets 

Block  Diagram 

These  components  interact  as  shown  in  the  block  diagrams  of  section  3  (refer  to  Figures  7 
and  8). 

Components 

TARGET  SET  TOOL 

The  Target  Set  Tool  is  the  GUI  for  direct  interface  with  the  system.  It  provides  a 
map  view  and  allows  the  user  to  search  for  and  select  facilities.  The  initial  version  was 
implemented  before  we  rigorously  followed  the  software  process  and  does  not  have  a 
design  document. 

COORDINATOR 

The  Coordinator  performs  target  system  analysis.  It  coordinates  flows  between  several 
models,  using  the  micro-economic  model  described  in  Section  2. 

OPTION  GENERATOR 

The  Option  Generator  proposed  direct  targets  to  achieve  a  set  of  desired  effects.  These 
are  displayed  in  the  Target  Set  Tool  for  the  user  to  approve. 

Interfaces 

There  are  two  external  interfaces  to  the  Endstate  system.  These  are  with  the  SDT 
Plan  Editor  and  with  the  MIDB. 

SDT  PLAN  EDITOR  INTERFACE 

The  Coordinator  communicates  with  the  Plan  Editor  by  XML  messages.  The 
Option  Generator  receives  XML  messages  from  the  Plan  Editor  and  sends  XML 
messages  to  the  Target  Set  Tool.  The  Target  Set  Tool  communicates  with  the  SDT 
through  direct  function  calls,  passing  Java  objects. 
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PLAN  EDITOR/COORDINATOR  INTERFACE  DATA 


Message 

Description 

PLAN 

Relevant  subset  of  the  plan  stored  in  the  SDT 

TARGET 

The  direct,  indirect,  no  strike  and  no  effect  targets 

AUGMENTATION 

Repeats  the  plan  message,  augmented  with  information  from  the 

TSA  simulation 

PLAN  EDITOR/OPTION  GENERATOR  INTERFACE  DATA 


Message 

Description 

PLAN 

Relevant  subset  of  the  plan  stored  in  the  SDT 

TARGET 

The  direct,  indirect,  no  strike  and  no  effect  targets 

OPTION  GENERATOR/TARGET  SET  TOOL  INTERFACE  DATA 


Message 

Description 

PROPOSED 

TARGET 

List  of  targets  proposed  by  the  algorithms  and  links  to  the  plan. 

COORDINATOR/TARGET  SET  TOOL  INTERFACE  DATA 


Message 

Description 

SOLUTION 

Simulation  results  for  display  in  Map  Tool. 

TARGET  SET  TOOL/PLAN  EDITOR  INTERFACE  DATA 


Message 

Description 

Collection  of 

facilities 

Facilities  that  are  on  the  direct,  indirect,  no  strike  or  no  effect  lists. 

Facilities  Event 

Facilities  selected  by  the  user  on  the  direct,  indirect,  no  strike  or  no 

effect  lists. 

ACCEPTED 

TARGET 

List  of  targets  and  links  to  the  plan,  proposed  by  the  algorithms  and 

accepted  by  the  user. 

31 


6  JEFX  04  SUPPORT 

6.1  Summary  of  EBO  EndState  Results  from  JEFX 


EBO  EndState-JEFX04  Environment  Conditions 

EBO  EndState  capabilities  were  provided  at  JEFX04  as  part  of  the  overall  EBO  initiative 
conducted  within  the  strategy  cell  division.  Since  targeting  decisions  are  not  the  function 
of  the  strategy  cell,  the  extent  to  which  EBO  EndState  capabilities  could  be  used  or 
evaluated  was  limited  by  the  process  hierarchy  conducted  within  the  strategic  planning 
operations.  EndState  capabilities  were  further  limited  by  data  availability.  Recall,  EBO 
EndState  provides  targeting  support  and  analysis  for  Effects-Based  Planning  Operations 
in  three  distinct  forms:  automated  target  option  generation,  target- systems  analysis  and 
manual  (map  and  or  query  based)  target  identification  and  selection.  As  stated  above,  the 
automated  targeting  capability  (Option  Generation)  and  the  Target  Systems  Analysis 
(TSA)  capabilities  require  sufficient  Enemy  Order  of  Battle  information  in  order  to 
execute  the  algorithms.  Data  availability  limitation  for  various  JEFX  events  was  as 
follows: 

In  Spiral  2,  the  link  data  was  sparse.  There  was  good  connectivity  between  producers 
and  transshipment  facilities  within  each  network.  However,  consumer  facilities  were 
missing  and  cross-network  links  were  absent.  As  a  result,  TSA  was  unable  to  function 
and  Option  Generation  was  required  to  work  in  a  limited  mode. 

For  Spiral  3,  many  of  the  POL  facilities  were  repositioned.  As  a  result,  many  links  would 
cross  much  of  Califon  and  reaching  a  particular  node  might  traversing  a  path  much  longer 
than  the  dimensions  of  Califon.  Additionally,  there  were  difficulties  configuring  the 
software  since  we  had  only  one  CPLEX  license  and  multiple  machines  that  would  have 
wanted  to  use  it.  Eventually,  we  were  able  to  use  Option  Generation  well  using  the 
nodes-only  capability. 

At  Main  Ex,  several  different  units  were  used  for  the  capacity  values.  As  a  result,  the 
network  was  artificially  constrained  to  have  very  little  flow.  The  issue  with  the  long  links 
from  Spiral  3  remained.  (This  did  not  cause  problems  for  the  software,  but  it  did 
represent  an  unrealistic  network.  As  a  result,  some  results  were  counter-intuitive  if  one 
assumed  a  reasonable  structure  for  the  networks.) 
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EBO  EndState  Training  Results 

As  part  of  each  JEFX  Spiral  and  Main  Ex.,  users  and  assessors  received  training  on  EBO 
EndState  capabilities  as  part  of  the  SDT  training  activities.  Lessons  learned  from  the 
JEFX  training  sessions  point  to  the  need  for  more  individualized  training  and  a  more 
favorable  training  environment.  SDT  training  for  JEFX04  was  conducted  in  a  noisy 
environment  with  trainees  attempting  to  follow  training  directions  via  a  headset 
communication  system.  A  typical  training  class  consisted  of  10-15  trainees  under  the 
direction  of  a  single  tutor.  While  ALPHATECH  provided  additional  training  facilitators 
to  answer  questions,  it  was  difficult  for  participants  to  catch-up  with  the  training  after 
suffering  a  fall-back.  In  addition  to  high  trainee/tutor  ratios,  there  was  an  issues  or 
participant  interest.  While  some  trainees  required  instruction  on  use  of  the  entire  system, 
most  users  were  focused  on  a  specific  task  or  function  of  the  software.  This  made  it 
difficult  to  hold  the  interest  of  the  trainees.  Future  training  sessions  would  benefit  from 
more  individualized  training  geared  toward  smaller  groups  and  focused  on  activities  that 
would  be  conducted  by  the  group. 

EndState  JEFX  Execution  Results 

While  EBO  EndState  capabilities  were  limited  by  the  absence  of  sufficient  link 
data  at  JEFX04  overall  assessment  of  the  product  was  very  positive.  Individual 
demonstrations  of  EBO  EndState  capabilities  were  presented  to  various  members  of  the 
Targeteering  contingency.  Demonstrations  included  an  overview  of  unavailable 
capabilities  for  automated  target  generation  and  analysis  as  well  as  Node  based  Option 
Generation  and  Manual  Targeting  capabilities.  Feedback  from  targeteers  drew  positive 
comments  in  that  EndState  capabilities  reiterated  many  of  the  capabilities  of  NASIC’s 
Tel-scope  Tool.  In  addition,  EndState  capabilities  were  praised  for  their  ability  to  extend 
to  multiple  target  systems  whereas  Tel-scope  applies  to  communication  systems  only. 

High  level  VIP  demonstrations  of  EndState  capabilities  and  SDT  in  general  drew 
praise  from  several  sources.  Gen.  James  (Head  of  the  EBO  Panel  for  the  C4ISR  Summit) 
and  Col.  Casserino  were  given  demonstrations  of  the  entire  suite  of  EBO  tools  available 
at  JEFX04.  These  individuals  were  truly  impressed  with  the  full  scale  integrated  nature 
of  the  SDT  suite  of  tools  and  the  capabilities  of  EBO  EndState  in  particular.  Comments 
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by  Col.  Casserino  indicated  his  recognition  that  SDT  tool  such  as  EndState  and  COG 
Articulator  provided  “real  analysis  capabilities  under  the  hood”  as  opposed  to  other  EBO 
products  on  display  which  “presented  information  but  provided  no  additional  analysis”. 
The  capability  to  identify  and  analyze  targets  using  EndState  capabilities  and  then  import 
those  targets  directly  into  the  SDT  plan  was  viewed  as  a  key  facilitator  of  Effects-Based 
planning. 

The  JEFX04  Main  Ex.  Branch  planning  activity  was  the  first  opportunity  to  use  EBO 
EndState  capabilities  as  part  of  the  experimental  process.  While  we  reiterate,  targeting 
was  not  a  primary  focus  of  the  branch  planning  effort  in  the  strategy  planning  cell,  there 
was  interest  in  viewing  and  assessing  the  EndState  capabilities  for  target  generation.  As 
part  of  the  branch  planning  efforts,  participants  Wing  Commander  Red  Thompson  and 
Tim  Spath  along  with  Maj.  Dietrich  and  Maj.  Greathouse  developed  plans  to  limit  enemy 
capabilities  in  area  X  (detailed  left  out  for  classification  purposes).  A  critical  desired 
effect  for  this  plan  included  denial  of  transport  of  materials  to  the  area  around  X.  These 
participants  employed  EndState  manual  targeting  query  and  map  capabilities  to  filter 
targets  to  display  road  and  rail  networks  in  the  vicinity  of  X.  They  were  further  able  to 
identify  (through  MIDB  queries)  chokepoints  to  effectively  cut  supply  routes  into  X. 
Once  identified,  they  exported  selected  targets  into  their  plan  under  the  appropriate  effect 
artifact.  These  capabilities  were  viewed  as  very  easy  to  use  and  beneficial  for  a  seamless 
effects-based  planning  process. 

Through  these  trials,  EBO  EndState  was  assessed  in  a  positive  manner.  There  was  a 
general  sentiment  that  EBO  EndState  capabilities  would  be  a  valuable  asset  to  targeteers 
at  future  JEFX  events.  The  primary  criticism/suggestion  to  enhance  EBO  EndState 
capabilities  centered  around  the  map  display.  There  was  a  sentiment  among  users  that 
transitioning  the  EndState  map  display  to  the  Falcon  View  mapping  system  would 
significantly  increase  its  utility. 
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7  DARPA  ENDSTATE  WORK 


7.1  Introduction 

DARPA  directed  ALPHATECH  to  investigate  approaches  to  understating  the  effects  of 
operations  as  it  pertains  to  the  planning,  execution,  and  assessment  of  military  operations. 
This  work  took  place  in  three  distinct  efforts,  chronologically  listed:  Endstate,  EBUNT, 
and  FogLight.  This  summary  focuses  on  the  FogLight  activity,  as  it  is  a  culmination  of 
the  lessons  learned  from  the  first  two  volumes  of  work.  As  deliverables,  we  provided 
over  two  dozen  separate  PowerPoint  presentations,  consisting  of  hundreds  of  slides  to  the 
DARPA  Program  Managers  (PMs).  As  such,  this  section  will  summarize  the  literature 
reviewed  and  analysis  performed  in  the  construction  of  the  briefing  materials  provided  to 
DARPA.  The  final  EBUNT  and  FogLight  presentations  are  provided  in  appendixes  D 
and  E  respectively.  These  presentations  effectively  summarize  the  results  of  all  three 
Efforts. 

7.2  Problem  Refinement 

At  the  Direction  of  our  DARPA  PM,  Mark  Greaves,  we  focused  our  2004  COG  modeling 
efforts  on  the  cognitive  aspects  of  planning,  execution,  and  assessment.  Leveraging  the 
previous  Endstate  work  in  reduced  ordered  modeling  and  time  scale-scale  decomposition, 
we  developed  the  FogLight  concept,  highlighting  the  role  of  human  decision  making. 

FogLight  addresses  the  adaptation  of  the  endstate  analysis  into  information  presentations, 
which  promote  quick  and  accurate  command  decisions.  As  evidenced  in  the  January 
2004  Future  Combat  System  (FCS)  Command  and  Control  (C2)  experimentation  at  the 
CASCOM  Battle  lab,  the  warfighter  is  hard  pressed  to  interpret  composite  endstate 
models  and  projected  scenarios  in  real-time  with  current  C2  displays.  Mark  Greaves 
identified  the  endstate  visualization  problem  as  the  most  crucial  part  military  planning, 
execution,  and  assessment,  because  the  final  decisions  drive  the  modeling  and  algorithm 
development.  Because  battlefield  stress  and  the  “fog  of  war”  make  traditional  User 
Interfaces  (UI)  ineffective,  the  warfighter  needs  interfaces  capable  of  presenting  the  most 
salient  portions  of  a  model  and  predicted  effects  into  his  current  frame-of-reference. 
Therefore,  the  FogLight  work  focuses  on  Warfighter  Machine  Interfaces  (WMIs)  that 
adapt  endstate  models  to  the  context  and  the  individual. 
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The  FogLight  vision  is  to  create: 

•  Decision  support  tools  customizing  themselves  to  user,  role,  device,  and 
Knowledge  Base  (KB),  resulting  in  faster,  more  accurate  decisions  in  unfamiliar 
contexts 

•  Effective  UIs  for  dynamically  composed  C2  software  systems 


The  FogLight  approach  is  to: 

•  Apply  recent  advances  in  diagrammatic  reasoning  research  to  develop  a  system 
that  dynamically  generates  user-specific  interfaces  (UIs)  according  to  each  user’s 
role,  cognitive  style,  and  available  display  hardware 

•  Develop  a  principled  means  for  combination  of  UI  elements,  mapping  key  domain 
dimensions  into  appropriate  interactive  displays,  and  preserving  the  most  valuable 
information  in  a  readily-understood  form 

•  Combine  the  advantages  of  current  paper-based  planning  collaterals,  such  as  maps 
and  sketches,  with  automated  inference  processes  that  dynamically  generate 
interface  displays  based  on  knowledge  of  the  task  and  domain 

The  FogLight  concept  anticipates  the  following  benefits: 

•  Improved  human  cognitive  capabilities  across  all  applications 

o  Reduced  time  on  task  by  compressing  inference  steps,  Example:  search  as 
direct  perception 

o  Reduced  user-system  interactions.  System  generates  interfaces  that  are 
likely  to  contain  the  internal  structure  of  the  solution 

o  Improved  support  for  complex  coordination  across  multiple  battlefield 
functions.  Example:  Dynamic  joint  replanning  involving  logistics, 
communications,  air-strike,  and  ground  forces 

•  Shortened  development  timelines  by  raising  the  level  at  which  GUIs  are  specified 

•  Lowered  barriers  to  adoption  of  automated  C2  systems 

o  Reduced  training  time.  Direct  encoding  and  task-based  UI  generation 
lower  learning  curves. 

o  Reduced  training  costs  and  staff  requirements  for  new  systems 

7.3  Concept  of  Operation  Development 

With  the  new  emphasis  on  model  abstraction  for  effective  decision  making,  we  were 
tasked  to  investigate  a  wide  variety  of  C2  domains,  best  illustrating  the  benefits  of 
technological  leaps  in  this  area.  We  produced  a  series  of  briefings  which  presented 
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potential  military  problems,  operational  challenges,  and  anticipated  impact.  The  C2 
applications  include: 

•  Diagrammatic  Coordination  of  Urban  Operations 

•  Sketch-based  Situation  Monitoring  and  Replanning 

•  Planning  and  Control  of  FCS  Robotic  Assets 

•  Using  Diagrams  to  Find  the  Right  Questions  for  Intelligence  Analysis 

•  Diagrammatic  Special  Ops  Planning 

•  Airborne  Command  and  Control  (AW ACS) 

•  Logistics  Routing 

•  Weapon  Target  Pairing 

See  Appendix  A  for  the  problem  descriptions  and  reasoning  approaches  for  these 
application  domains. 

By  taking  some  of  the  most  challenging  attributes  of  each  domain,  including  dynamic 
replanning,  handheld  devices,  battlefield  stress,  and  rapid  decision  making,  we 
formulated  a  detailed  scenario  to  help  guide  the  modeling  and  algorithmic  framework. 
We  combined  field  reports  from  Operation  Anaconda,  Operation  Iraq  Freedom,  and  field 
manuals  to  construct  a  use  case  for  the  FogLight  system:  Non-Regular  Enemy  Force 
Attacks  Exposed  Logistics  Operations  (see  Appendix  B  for  several  slides  illustrating 
the  scenario  using  operational  graphics).  We  gathered  several  screenshots  from  many 
current  and  Next  Generation  C2  systems,  including  the  following: 

•  TBMCS  air  operations  planning  and  execution 

•  FCS  Warfighter-Machine  Interface  current  concepts 

•  BCS3  next  generation  Logistics  situation  awareness,  planning,  etc 

•  FCS  Unit  of  Action  Communications  Networks  concepts 

We  analyzed  these  screens  for  cross-COG  models,  adversarial  representation,  human 
decision  points,  input  requirements,  state  estimation,  simulation,  and  projected  impact 
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capabilities.  The  FCS  Warfighter  Machine  Interface  (WMI)  requirements  stressed  the 
following  constraints  on  display  of  information: 

-  WMI  system  generates  all  GUIs  from  UI  specs  passed  by  mission  applications 

-  Tailor  displays  to  user  role,  authorization,  and  preference 

-  Field  operation  in  C2  vehicles,  finger  selectable,  non-cluttered  displays 

-  Common  look-and-feel  across  all  UIs 

-  Warfighter  centric 

These  WMI  requirements  proved  quite  useful  in  framing  the  visualization  architecture  for 
Endstate  models  and  algorithmic  results.  From  the  CASSCOM  Battlelab  FCS  C2 
experimental  results  we  identified  a  number  of  outstanding  needs: 

-  Enhanced  visualization  and  dissemination  of  the  tactical  scheme  of  maneuver 
enables  effective  collaboration  (FCS  C2  req) 

-  Decision  tools  need  support  integrated  battlespace  management  of  the  battlefield 
functional  areas  (FCS  C2  req) 

-  Tactical  decision  making  requires  a  running  estimate  that  provides  a  continual 
flow  of  actionable  information,  predictive  intelligence,  visualization,  and 
presentation  capabilities  (FCS  C2  req) 

We  combined  the  analysis  of  existing  systems,  FCS  WMI  requirements,  and  outstanding 
operational  needs,  into  a  detailed  scenario  specifying  Army  Universal  Task  List  (AUTL) 
tasks  performed  by  a  number  of  military  component  in  our  identified  use  case.  In  Figure 
7,  these  steps  are  organized  into  categories  of  “Observe,  Orient,  Decide,  and  Act,” 
otherwise  known  as  the  OODA  loop. 
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1 .  Supply  unit  issues  “under  lethal  attack”  alert 

2.  Mobile  Command  Group  (MCG1)  receives  alert 

3.  MCG1  staff  requests  location  clarification  and 
battle  readiness  info  not  in  log  display 

4.  MCG1  staff  passes  alert  to  MCG1  commander 

5.  MCG1  commander  sends  alert  to  UA  XO 

6.  MCG1  commander  compares  text  alert  messages, 
log  display,  and  tactical  display  and  orders  support 
platoon  in  position  to  engage  enemy  Co 

7.  Tactical  Command  Post  (TACP)  staff  receives 
alert  message 

8.  TACP  staff  passes  alert  to  Unit  of  Action  (UA)  XO 
in  the  Command  Integration  Cell  (CIC) 

9.  Unit  of  Action  (UA)  XO  sends  numerous  info 
requests 

1 .  ISR  tasking  to  acquire  enemy  Co 

2.  Status  of  air  assault,  air  lift,  close  air  support 
(CAS),  cavalry,  and  security  assets 

3.  Location  of  Fwd  Air  Controller  Planner 
(FACP)  and  impact  of  retasking 

10.  UA  XO  receives  mission  status,  location, 
readiness,  and  current  tasking  information 

11.  UA  XO  decides  to  use  an  available  AC-1 30 
gunship  (XCAS),  air  assault,  and  the  FACP; 
discovers  security  platoon  is  already  conducting 
“attack  to  defend” 

1 .  Tasks  security  platoon  to  locate  secure  drop 
point  for  air  assault  company 

2.  Orders  FACP  to  support  rescue  mission 


12. 

UA  command  staff  plans  mission 

12. 

Searches  for  all  relevant  units  in  Order  of  Battle 

13. 

Develops  air  assault  mission 

12.  Tasks  air  assault  Co  to  “attack  to  fix”  enemy 

company 

1 3.  Assigns  air  assault  Co  commander  as  mission 

commander 

14.  Attaches  air  strike  helos  and  FACP  to  Co 

1 5.  Tasks  closest  Medic  Unit  to  triage  casualties 

13. 

AOC  creates  tasking  for  XCAS 

12. 

Identifies  kill  box 

13. 

Requests  airspace  control  measures 

14. 

Deconflicts  airspace 

15. 

Sends  JTIDS  tasking  message  to  retask  AC-130 

16. 

Establishes  communications  frequencies 

14. 

UA  command  staff  sequences  the  timing  among  Army  and 

Air  Force  assets 

12. 

Security  platoon  holds  defensive  position  until  1345 

13. 

Air  strike  drops  troops  at  1300 

14. 

Air  assault  fixes  enemy  by  1 31 5 

15. 

FACP  send  target  coordinates  9-line  at  1320 

16. 

XCAS  attacks  to  destroy  at  1 325 

17. 

Air  assault  runs  clean  up  at  1345 

18. 

Medics  triage  casualties  in  log  transfer  point 

□Observe  □Orient  □  Decide  I  Act 


Figure  9  -  These  steps  detail  the  tasks  performed  by  a  number  of  elements  during  a  joint  operation 
where  dynamic  replanning  is  required  due  to  an  unanticipated  flank  attack. 

By  constructing  these  steps,  we  were  able  to  construct  metrics  for  measuring  replanning 

activities  with  and  without  a  FogLight-type  systems  (an  explanation  of  the  metrics 

framework  is  covered  in  the  attached  FogLight  briefing).  The  technological 

underpinnings  for  a  FogLight  concept  are  described  in  the  next  subsection. 

7.4  Technical  Results 

With  the  FogLight  approach  and  detailed  use  case  defined,  we  refined  our  modeling 
approach  by  elaborating  the  cross-COG  model  transformation  steps  underlying  each  step 
in  the  scenario.  This  work  resulted  in  a  literature  study  of  relevant  fields,  development  of 
a  FogLight  architecture,  and  the  construction  of  candidate  UIs  resulting  from  the  various 
transformations. 

The  approach  to  FogLight  draws  from  fields  such  as  Diagrammatic  Reasoning, 
Geographic  Information  Systems,  dynamic  control  theory,  collaborative  agent 
architectures,  knowledge-based  systems,  cognitive  science  studies  (pre-attentive 
recognition),  and  heterogeneous  logics.  See  Appendix  C  for  a  summary  of  the  findings 
from  our  literature  survey  on  model  abstraction  for  effective  decision  making.  We 
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constructed  a  broad  program  architecture  for  FogLight  in  which  technological 
components  can  be  developed.  The  FogLight  system  architecture  is  depicted  in  Figure 
10. 


Qualitative  spatial  reasoning 
Diagrammatic  plan  representation 
Automated  diagrammatic  reasoning 
Knowledge-based  inference 


Qualitative  and 
Diagrammatic 
Reasoning 


•  Knowledge-based,  probabilistic 
information  needs  generation 
'  Information  integration  from 
heterogeneous  data  sources 
'  Temporal  projection 


Incremental 

Qualitative 

Simulation 


k  \ 

User,  Role,  Device, 
Reasoning  Mode 


Intelligent  Interface 
Generation 


1  Intelligent  information  filtering 
'  Best-first  information 
transmission 

'  Automated  encoding  of  problem 
structure  in  diagram 
1  Tailoring  of  information 
presented  to  each  individual 
■  Mediation  of  inter-service 
communications  via  information 
preserving  diagrammatic 
transformations 


Figure  10  —  The  FogLight  system  architecture  consists  of  three  general  components:  1)  the 
Qualitative  and  Diagrammatic  Reasoning  module  performs  state  estimation  to  reason  about  plan 
sketch  elements,  2)  the  Qualitative  Simulation  module  is  responsible  for  prediction  and  causal 
reasoning,  and  3)  the  Intelligent  Interface  Generation  module  transforms  the  COG  models  into  UIs 

by  emphasizing  the  most  salient  effects. 

The  FogLight  system  architecture  is  intended  to  help  develop  a  DARPA  program  to  build 
a  context-aware  information  system.  Conventional  interfaces  are  minimally  aware  of  the 
device,  user  profile,  current  task,  and  critical  data  elements.  There  is  a  lack  of  scientific 
foundation  for  the  effective  dynamic  generation  of  visualization  and  UI  elements.  The 
FogLight  concept  focuses  on  role-specific  interfaces,  as  shown  in  Figure  11,  which 
produce  plan  sketches,  situation  monitoring,  command  direction,  and  predicted  outcomes 
for  an  individual  context. 
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•  User’s  characteristics 

-  Role 

-  Prefers  tabular  data 

•  Device 

-  Size  of  display 

-  Touch-screen  capability 

•  Reasoning  context 

-  Interrupfibility 

•  Data  sources 

-  Speed  of  comms 

-  Quality,  quantity  of  data 


Dynamic  Interface  Constructor 


duui  cco  ' - t t - 1 — | — 

eed  of  comms  link  /  /  Tantinall 
lality,  quantity  of  data  /  /  ' dU/ . 

//  situation^  I 

Broad  //  \7 

overview f  *  -- 


Immediate 

direction 


Unit  of  Action  Executive  Officer 


Figure  11  — The  FogLight  architecture  uses  simulation,  diagrammatic  reasoning,  and  UI  generation 
to  produce  role-specific  interfaces  for  different  echelons  and  tasks. 

The  technological  challenge  presented  by  the  FogLight  architecture  is  to  determine  the 

principles  for  mapping  situation  data  to  interfaces.  In  essence,  finding  the  appropriate 

model  abstractions  and  transformations  between  terrain  and  tracking  data,  plan  and  task 

orders,  and  visualization  and  GUI  compositions.  We  decomposed  this  problem  into  three 

constructive  research  goals  reflecting  the  system  architecture: 

1 .  Provide  a  basis  for  constructing  a  general,  provably  correct  system  of 
information  transformations,  forming  the  basis  of  each  architectural 
component,  by  exploiting  advances  in  logic  for  describing  mappings  between 
semiotic  systems. 

2.  Determine  the  choice  among  otherwise  valid  structures  based  on  current 
context,  by  applying  recent  findings  from  cognitive  research  on  human 
perception. 

3.  Develop  general  principals  for  mapping  situation  data  to  useful  interfaces, 
committing  to  a  recognizable  structure,  composing  UI  elements  (e.g. 
windows,  menus,  toolbars,  etc),  and  formats  (e.g.  tables,  bar  charts,  graphs, 
etc.),  by  applying  advanced  UI  research  in  task  models  and  device 
independent  GUI  frameworks. 

We  also  developed  a  candidate  functional  architecture,  pictured  in  Figure  12,  to  illustrate 
the  information  flow  of  a  FogLight  context-aware  information  system. 


41 


Figure  12  —  The  FogLight  functional  architecture  shows  the  information  flow  of  a  context-aware 
systems  with  processing  stages  shown  as  yellow  ovals  and  output  artifacts  shown  as  green  boxes. 

FogLight  replies  on  a  specific  formulation  of  Situational  Awareness  (SA)  to  ground  its 

models  in  psycho-physiological  limitations  of  human-computer  decision  making  and  a 

hybrid-reasoning  model  which  combines  diagrammatic  and  propositional  models  to 

ground  FogLight  transformations  in  logic  formalisms. 

First,  we  summarize  our  model  of  Situational  Awareness.  The  FogLight  SA  model, 
including  on  the  latest  version  of  the  Unit  of  Action  Operational  and  Organizational  Plan, 
builds  on  the  Endsley’s  approach  to  mental  modeling: 

-  Perception  of  elements  in  the  environment  within  a  volume  of  time  and 
space,  include  number,  location,  and  capabilities  of  all  enemy  and  friendly 
forces  in  a  given  area  and  their  relationship  to  other  points  of  reference. 

-  Comprehension  of  elements  meaning,  the  capability  to  identify  patterns 
and  develop  an  integrated  picture  of  the  environment  -  the  first  level  at 
which  the  decision-maker  moves  from  awareness  to  understanding. 

-  Projection  of  their  status  in  the  future;  the  definition  of  the  future  actions 
of  the  perceived  elements  in  the  environment. 

-  Technical  (What’s  there  and  what  is  not  there) 

•  Location.  Where  is  the  entity?  Perceived  truth  (unusable, 
actionable,  targetable) 

•  Acquisition.  What  is  the  entity?  (detected,  classified,  and  id’d) 

•  State.  Entity  health?  Mission  capable  status  (alive,  hit,  dead) 

-  Cognitive 
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•  Location  relative  to  points  of  interest  (blue/red,  civilians,  built  up 
areas,  etc.) 

•  Acquisition.  What  are  they  capable  of  doing  (range  fans,  munition 
footprint,  etc) 

•  State.  Entity  health  and  weapon  status  (fully  armed,  fueled, 
disrupted,  etc) 

Next,  we  present  our  hybrid  reasoning  model,  in  Figure  13,  which  builds  on  the  research 
of  Goguen  and  Harrell  (2003),  Garagani  and  Ding  (2003),  and  Barwise  and  Etchemendy 
(1995). 
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Figure  13  -  The  hybrid  model  reduces  inference  scalability  short  comings,  such  as  in  the  ramification 
problem,  by  linking  propositional  logic  with  analogical  reasoning  via  transformations  among  linked 

views. 

Hybrid  reasoning  focuses  on  methods  for  combining  diagram  (analogical)  representations 
and  propositional  descriptions,  which  can  ground  to  provable  semantics  for  a  description 
of  information  equivalence.  Some  of  the  most  promising  work  is  done  by  Garagnani 
(2003),  who  has  derived  a  formal  hybrid  framework,  to  obtain  a  2  to  90  times  speed  up 
over  classic  planning  approaches.  This  is  the  theoretic  foundation  for  FogLight’s  context- 
aware  computing.  In  the  next  subsection  we  present  the  results  of  using  hybrid  reasoning 
as  applied  to  the  problem  of  generating  dynamic  UIs. 
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Example  FogLight  Transformations 

FogLight’s  Dynamic  UI  Generation  draws  from  a  body  of  work  on  automated  generation 
of  visual  information,  known  as  automated  design.  MacKinlay  is  the  progenitor  of  this 
field,  defining  the  concepts  of  expressiveness  and  effectiveness,  but  work  continues  today 
leading  into  hybrid  reasoning  through  the  analogical  representations  (discussed  in  the 
previous  subsection). 

Here  we  present  a  series  of  candidate  information  displays  that  result  from  the  model 
transformation.  These  examples  attempt  to  illustrate  the  challenges  of  automated  design, 
particularly: 

•  Displays  perform  information  preserving  transformations  among  linked  views 
enabling  trade-off  and  constraint  enforcement 

-  Sketches  and  diagrams  bridge  communication  gaps  between  military 
domains  (Army,  Air  Force,  Log,  ISR,  Comms,  etc) 

-  Implied  tasks  and  information  requests  are  inferred  from  UI  elements 
through  qualitative  reasoning 

•  Aggregation  of  models  produces  visual  information  that  considers  the  viewers’ 
role,  task,  and  cognitive  situational  context  to  assess  the  value  ratio  of  information 

-  User  Modeling 

-  Cognitive  Task  Analysis 

•  Hybrid  representational  frameworks  automatically  synthesize  qualitative 
(diagrammatic/view)  models  from  varied  data-rich  domains  through  spatial  and 
topological  composition  processes 

•  Robust  Automation:  Nothing  gets  missed 

-  Provable  equivalence  ensures  the  accuracy  of  abstraction  steps 

-  Displays  are  adapted  by  programming  by  example  learning  techniques 

The  first  display  (Figure  14)  shows  the  type  of  information  that  may  be  displayed  to  an 
executive  commanding  (XO)  officer  during  a  dynamic  replanning  activity. 
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Convert  search  for  information  into  direct  perception 


Situation  display 
reconfigures  to  highlight 
the  most  relevant 
elements  for  the  replan. 
All  units  incapable  of 
participating  in  the 
replan  are  grayed, 
creating  a  mission- 
capable  situation 
awareness 


The  ISR  button  is 
added  as  a  result  of 
reasoning  about  the 
event  severity  replan 
activity  and  time  scale 


Time  constraint  translated  into  geographic  question, 
"who's  closest?*  XO  looks  for  the  closest  units  with  the 
right  capabilities  to  establish  the  response  time  frame. 


Select  Air  Force 
control  measures 
are  brought  into  the 
Army  replanning 
view,  highlighting 
the  kill  box  of  the 
AC-130 


User  gestures  are 
confined  to 
movements  which 
result  in  feasible 
plans  via  constraint 
enforcement 


Figure  14  -  The  Unit  of  Action  executive  commanding  officer  (XO)  information  display  aggregates 
multiple  situation  models,  mapping  to  his  current  frame  of  reference  (based  on  his  latest  plan). 

This  display  utilizes  a  board-base  of  command  experience,  where  decisions  are  often  a 

quick  recognitional  process,  like  a  chess  master  choosing  from  patterns  of  moves.  On  the 

other  hand,  a  newly  trained  foot  soldier  may  need  some  of  the  same  information,  but  he 

must  make  quick  decisions  with  less  experience,  so  he  need  tasks  clearly  displayed  with 

contextual  guidance. 
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Exploit  unique  perceptual  and  motor  skills  of  the  user  to  compensate 


Splitting  visual 
attention  to  a 
calculated 
threshDlc 
capitalizing  on 
user's  peripheral 
perception  skills. 


Diagram  graphics 
are  isomorphic  to 
the  relationships  of 
objects  in  the  real 
world.  The  user  is 
guided  to  the  most 
LseHjl  asses,  and 
cecision  making 
processes  using 
spatial  and  analytic 
reasoning.  The 
interface  exploits 
excep:ionalfine 
motor  skills. 


The  steps  of  the 
cecision  process  are 
cisplayed  as  a  graphic 
to  orient  :he  novice  user 
in  the  overall  process. 


Tne  sit_ia:ion  graphic 
d  splay  uses  direct 
encoding  ofdiagram 
eements  draped  over  a 
sensor  view,  leveraging 
the  user's  skill  in 
navigating  3D  worlds, 
de- emphasizing  weakly- 
learned  operational 
graphics,  ard  supporting 
the  current  task. 


Time  constraint  is  directly  encoded  as  a  countdown  timer 
irnp-essiny  lie  estimated  safe  task  execution  lime  where 
assets  are  like  not  to  be  comp'onised 


Figure  15  -  A  dismounted  soldier’s  display  is  tailored  to  his  modal  preferences  and  current  tasks, 
compensating  for  his  limited  experience  with  operational  graphics. 

Though  the  foot  soldier  is  not  thinking  strategically,  like  the  XO,  he  still  requires  the 

appropriate  context  to  carry  out  the  spirit  of  the  commands  if  something  unexpectedly 

goes  wrong.  In  a  different  vein,  coordination  of  joint  operations  is  particularly 

challenging,  as  symbology,  information  displays,  and  task  decompositions  vary  among 

the  forces.  Figure  16  shows  an  example  of  a  display  which  synchronizes  the  information 

across  military  branches. 
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Joint  re-planning  is  assisted  through  the  automatic  tailoring  of 
interfaces,  translating  ‘Army-speak’ into  familiar  AF  symbology. 


Situation  display 
reconfigures  into  a  I  nkec 
geographic  display  of 
kill-  boxes  paired  to  a 
table-based  temporal 
d  splay  as  a  function  of 
the  airspace 
management  task,  the 
short  cecision  cycle,  and 
the  air  planner's 
p-eference  for  tabular 
d  splays. 


The  critical  lime  frame 
for  nesting  Army 
requirements  is 
emphasized  tc  focus 
time  vs  resource  trade¬ 
offs. 


Time  eonstra  nt  translated  into  spatial  region  in  which 
:asks  must  be  shifted  to  meet  new  requirements  while 
minimizing  consecuent  effects  on  deoeidenl  tasks. 


Ar  Force  control 
measures  aid 
pertinent  assets  are 
included  in  tie 
display  maiitaining 
a  consistent  view 
and  highlghting  the 
kill  box  of  the  AC- 
1 30  to  aid  re¬ 
planning. 


The  taou  ar  display 
is  synthesized  with 
diagrammatic 
elements, 
maintaining  direct 
encodng  of  the 
domain  objects  and 
their  important 
relaionships,  in  tiis 
case  mission 
duration,  circle 'ing, 
and  depencercy. 


Figure  16  -  An  Air  Force  planner’s  information  display  translates  Army  operational  graphics  and 
chooses  a  gantt  chart  for  the  display  of  temporal  information. 

7.5  Conclusion 

The  rapid  increase  in  capabilities  and  complexity  of  current  and  anticipated  C2  systems 
will  place  comprehension  of  planned  (endstate)  effects  on  the  critical  path  to  designing 
decision  support  interfaces.  Recent  research  has  generated  promising  results  regarding 
effective  hybrid  model  abstraction  techniques  for  state  estimation  and  prediction,  and 
cognitive  science  studies  of  human  perceptual  biases.  ALPHATECH  has  development 
the  FogLight  concept,  at  the  direction  of  the  Endstate  DARPA  PM  Mark  Greaves,  to 
establish  the  technical  foundations,  detailed  use  cases,  and  anticipated  results  from 
advancements  in  COG  modeling,  cross-COG  transformations,  and  prediction  of 
adversarial  reactions  and  effects.  ALPHATECH  created  a  number  of  briefings  to  specify 
scenarios,  architectures,  candidate  UIs,  and  a  metrics  framework  to  show  how  FogLight 
can  go  beyond  the  current  state  of  practice,  where  systems  only  accommodate  limited 
user  and  device  variance,  GUIs  are  hand  crafted,  and  cognitive  task  modeling  is  labor 
intensive.  The  envisioned  FogLight  system  will  result  in  more  accurate  decisions  under 
duress,  lower  the  barriers  to  adoption  of  automated  C2  systems,  and  reduce  the 
development  time  for  more  effective  interfaces. 
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7.6  Background 

The  first  Endstate  work  began  by  looking  into  promising  model  abstraction  techniques, 
which  formed  the  foundation  for  FogLighf  s  approach  to  planning,  execution  and 
assessment  of  Effects-Based  Operations.  In  this  subsection  we  include  an  abstract  of  the 
Understanding  EBO:  Model  Abstraction  and  Achieving  a  Favorable  Endstate  briefing 
presented  by  Alan  Evans  at  the  Enabling  Technology  For  Simulation  Science  V 
symposium. 

Under  the  DARPA  Endstate  program,  ALPHATECH  has  examined  the  problem  of 
understanding  the  effects  of  operations  on  multiple,  complex  and  highly  interconnected 
networks  within  a  nation-state's  infrastructure.  The  unifying  technical  concept  of  much 
of  the  Endstate  work  is  model  abstraction.  Endstate  has  considered  two  basic  forms  of 
model  abstraction  in  connecting  models  of  different  systems.  The  first  is  reduced-order 
modeling,  an  inductive  approach  to  modeling  aggregated  problems  in  variable  spaces  of 
reduced  dimensionality.  The  second  form  of  model  abstraction  considered  is  time-scale 
decomposition,  leading  to  hierarchies  of  related  models,  in  what  might  be  called  a 
deductive  approach.  Model-based  abstraction  of  network  facilities  and  links,  together 
with  constraining  physics,  as  well  as  the  priorities  and  decisions  of  embedded  operators 
or  controllers,  are  planned  for  networks  of  interest. 

Endstate  work  has  been  based  on  the  following  premises:  (1)  that  models  of  critical 
systems  such  as  electric  power  generation  and  distribution,  transportation  networks, 
telecommunications,  etc.,  already  exist  at  levels  of  resolution  exceeding  the  foreseeable 
requirements  of  operational  and  strategic  planning,  (2)  modeling  dependencies  among 
these  sub-systems  requires  fundamental  advances  in  modeling  techniques,  and  (3)  adding 
adversary  reactions  and  workarounds  as  feedback  to  the  models  will  demand  a  flexible 
approach,  drawn  from  fields  such  as  dynamic  control  theory,  collaborative  agent 
technology,  neural  nets  and  knowledge-based  systems.  Prototype  work  of  some  of  these 
adversarial  adaptive  modeling  techniques  has  been  demonstrated  this  year  under  DARPA 
Endstate. 

Concept  development  is  also  underway  on  the  techniques  that  can  be  applied  to 
assessment  of  causality  and  option  generation  in  support  of  COA  analysis.  Daunting 
obstacles  to  progress  include  the  sheer  size  of  the  range  of  outcome  spaces  and 
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combinatorial  complexity  of  higher-order  effects,  and  the  fact  that  causality  may  not  even 
be  addressed  by  current  models  or  supporting  technologies.  In  the  longer  term,  the 
paradigm  of  model  abstraction  and  cross-connection  of  model  hierarchies  is  being 
applied  to  concept  development  and  prototyping  in  other  domains.  The  models  involved 
may  be  prototypes  of  system  components  being  considered  for  acquisition,  or  conceptual 
simulations  of  future  C2  and  C4ISR  systems. 
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APPENDIX  A:  FOGLIGHT  APPLICATION  DOMAINS 


This  section  includes  several  summary  slides  for  each  military  domain  in  which 
diagrammatic  reasoning  may  provide  substantial  operational  impact. 


Plan  Monitoring  and  Reasoning  about  Army  COA  Sketches 

•  Problem 

-  COP  does  not  provide  “big  picture”  for  good  decision  making.  The 

COP  is  a  large  DB  with  thousands  of  relationships,  streaming  input,  and 
largely  textual  or  quantitative  data.  Human  decision  makers  think  in  terms 
of  strategy,  tactics  and  plans. 

-  Humans  can  perceive  relationship  visually  better  than  forming  and 
scanning  DB  queries.  DB  integration  technology  handles  data 
distribution,  normalization,  and  relational  queries  but  is  not  well  suited  to 
present  key  information,  such  as  multidimensional  sources  of  change, 
which  human  can  easily  perceive  visually. 

-  Replanning  is  difficult. 

•  Approach 

-  Transform  situational  awareness  data  into  projections  onto  COA 
diagram.  [Larkin  &  Simon,  Barwise  &  Seligman]  Diagrams  automatically 
support  a  large  number  of  perceptual  inferences,  easy  for  humans  to  do; 
[Larkin  &  Simon]  Diagrams  ease  perception. 

-  Use  diagrammatic  comparisons  to  detect  significant  plan  deviation. 
[Barwise  &  Etchemendy]  Diagrams  can  be  used  not  only  as  a  heuristic 
aid,  but  also  for  critiquing.  Sound  diagrammatic  proof  systems  can  be 
constructed. 

-  Use  diagram-based  HCI  for  replanning.  [Davies  &  Forbus]  Diagram 
state  alone  is  sufficient  to  transform  analogous  solutions  to  current 
problem;  [Koedinger]  Use  of  visual  schemata  can  enable  reasoning  short 
cuts  in  problem  solving;  [Larkin  &  Simon]  locality  aids  search.  Reduction 
in  number  of  symbolic  matches  required  to  evaluate  options. 

Planning  and  Control  of  FCS  Robotic  Assets 

•  Problem 

-  Tasking  robots  is  significantly  more  complicated  than  tasking 
humans.  Planning  and  control  of  robotic  systems  involves  more  detailed 
commands  and  status  monitoring.  Robots  have  less  ability  to  deal  with  the 
unexpected. 

-  Teleoperation  doesn’t  work  and  doesn’t  scale.  Get  rid  of  teleoperation 
without  removing  the  visual  aspect  of  it.  The  reduced  visual  display  of  the 
typical  robotic  asset  make  it  difficult  to  understand  status  and  take  control 
actions. 

•  Approach 

-  Use  information  equivalence  to  transform  human  commands  and 
robotic  status.  [Ferguson  &  Forbus]  Various  general  low-level  perceptual 
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propositions  can  be  linked  with  domain  specific  rule.  High-level 
characteristics  can  be  inferred  to  perform  qualitative  spatial  reasoning. 

-  Explore  efficient  robotic  plans  via  diagrammatic  constraints. 

[Chandra]  Model-Instance  Based  Reasoning  done  with  diagrams  promote 
inductive  leaps  or  "warrants  for  generalization"  to  derive  general 
propositions  from  concrete  instances;  [Goel,  1995]  Human  problem 
solvers  exploit  diagram  vagueness  to  discover  effective  solutions;  [Davies 
&  Goel]  Visual  analogies  aid  problem  solving 

-  Project  diagram  elements  into  robotic  sensor  display.  [Davies] 

Diagram  state  alone  is  sufficient  to  transform  analogous  solutions  to 
current  problem;  [Koedinger]  Use  of  visual  schemata  can  enable  reasoning 
short  cuts  in  problem  solving 

Using  Diagrams  to  Find  the  Right  Questions  for  Intelligence  Analysis 

•  Problem 

-  Information  Overload  Intelligence  analysts  read  hundreds  of  reports  a 
day,  looking  for  patterns  of  activities  akin  to  searching  for  fragments  of 
needles  in  multiple  haystacks. 

♦  This  seems  like  more  of  visualization  problem.  This  type  of 
pattern  matching  is  not  trying  to  make  implicit  information 
explicit. 

-  Collaborative  analysis  is  complicate  by  distributed  reports  The  “big 
picture”  of  an  analytic  line  is  often  spread  over  a  multiple  of  shorter  inter¬ 
dependant  documents.  Analysts  often  have  no  time  to  write  a 
comprehensive  summary  analysis. 

-  Classification  management  and  source  tracking  is  time  consuming 

Because  an  analysis  is  distilled  into  text,  the  contributing  evidence  (and  its 
attributes)  is  not  wholly  retained.  Source  attribution  and  declassification 
activities  are  costly  because  no  robust  linkage  to  original  data.  This 
detracts  from  focusing  on  the  analysis. 

•  Approach 

-  Information  equivalence  enables  the  diagrammatic  construction  of 
analytic  queries  [Chandra]  Problem  solving  is  driven  by  the  most  relevant 
info  in  any  domain.  The  interplay  between  spatial  and  non-spatial 
predicates  enables  opportunistic  solution  discovery. 

-  Use  diagram-based  HCI  to  capture  an  analytic  line  [Yan,  Forbus, 
Gentner]  Theory  of  re -representation  in  match  making  through  detecting 
opportunities,  generating  suggestions,  and  strategies  for  controlling  re¬ 
representation. 

-  Spatial  propositions  contribute  to  the  generation  of  intelligence 
requirements. 

Diagrammatic  Special  Ops  Planning 

•  Problem  focus:  Reducing  the  time  to  construct  robust  plans  in  the  face  of 
compressed  timelines 

•  Afghanistan  Scenario:  Lead  time  for  SOF  missions  greatly  compressed  from 
doctrinal  planning  cycle  (96  hr  ->  24  hr)  which  could  lead  to  greater  mission  risk 
and  compromised  plan  quality 
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•  Defense  payoff :  Reduce  turnaround  time  between  JSOCC  MIT  ASK  generation 
and  SOF  MICON/MSR  development 

•  Technical  Challenge  :  Respect  feasibility  of  complex,  interlocking  planning 
constraints 

-  Spatial 

-  Temporal 

-  Resource 

-  Capability 

•  Enabling  Technology  :  Diagrammatic  reasoning  formalism 

-  Visually  encode  different  constraint  types  in  a  collection  of  diagrams 

-  User  views  and  constructs  plans  within  alternate  constraint  perspectives 
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APPENDIX  B:  DETAILED  USE  CASE 


Detailed  Use  Case:  Non-Regular  Enemy  Force  Attacks  Exposed 

Logistics  Operations 

Situational  Awareness  and  Dynamic  Replanning 
Enabled  through  Information  Equivalence 
Transformations  Among  C2  Displays 

Operation  Iraqi  Freedom-like  Scenario: 

Attack  to  Secure  OBJ  DOG,  Attack  to  seize  OBJ  DAD 
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Unanticipated  Progress:  Ground  attack  out 
pacing  air  support  and  logistics 


Air  support  is  stressed 
with  ops  too  far  fwd  for 
refueling  (need  to 
reconfigure  airspace 
lay  down);  Decide 
between  goal  of 
reducing  OBJ  DAD 
combat  power  to  > 
50%  OR  continuing  to 
secure  axis  thunder 
south  of  OBJ  DOG 


Maneuver  sustainment 
is  stressed  with  ops 
more  fwd,  consuming 
more  fuel  (stretched  log 
chain  near  breaking 
point);  security  area  is 
much  larger  than 
planned. 


Enemy  Attacks  Weak  Point  in  Support 
Operations 


Air  support  focused  on 
CAS  for  OBJ  DAD; 
refueling  obits  moved 
fwd  to  support  short 
range  of  A-lOs  during 
CAS. 


Enemy  Co.  of  non¬ 
regulars  attack  exposed 
logistics  unit.  Unit  must 
send  “under  lethal 
attack”  alert  to  UA 
mobile  cmd  group.  UA 
CIC  conducts  rapid, 
dynamic  replan  with 
closest  forces  (joint). 
Adjacent  Log  units 
must  perform  support 
functions  to  coordinate 
joint  response. 


Axis 
Bench 

Larkin  &  Simon  [87], 
Garagnani  [03] : 

Diagrams  can  organize  a 
problem  description 
according  to  the  internal 
structure  of  the  solution 
path. .  .thus  reducing  the 
number  of  (search)  steps 
needed  to  solve  a  problem, 
as  shown  in  two  sample 
search  problems. 
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APPENDIX  C:  LITERATURE  SURVEY 


ALPHATECH  was  directed  by  Mark  Greaves  to  develop  the  FogLight  concept 
leveraging  recent  advances  in  diagrammatic  reasoning,  cognitive  science  studies,  and 
heterogeneous  logics.  The  following  is  a  brief  summary  of  our  findings. 

•  Graphical  Excellence  (Tufte) 

-  Well-designed  presentation  of  interesting  data  -  a  matter  of  substance,  of 
statistics,  and  of  design 

-  Complex  ideas  communicated  with  clarity,  precision,  and  efficiency 

-  Givers  the  viewer  the  greatest  number  of  ideas  in  the  shortest  time  with 
the  least  ink  in  the  smallest  space 

•  Computational  Synthesis  of  graphical  presentations  (Mackinlay) 

-  Expressiveness  -  conveying  all  of  the  facts  and  only  those  facts 

-  Effectiveness  -  Use  of  visual  predicates  that  humans  are  better  at 
perceiving 

•  Small  multiples  (sometimes  referred  to  as  an  icon  or  graphic 
cluster  or  glyph) 

-  APT  system  uses  the  generate-and-test  method 

•  Semiotic  Morphisms  (Goguen) 

-  Preserve  structure  rather  than  content  when  a  trade-off  is  forced 

-  Preserve  high  level  types  over  ordering  rules  of  sign  construction 

•  Goguen  &  Harrell  2003 

-  Used  algebraic  abstract  data  type  theory  to  define  semiotic  morphisms 
providing  information  preserving  transformations  between  representations 

-  Applied  theory  of  info  visualization  to  measure  quality  of  several  concrete 
examples,  identified  general  principles 

•  Garagnani  &  Ding  2003 

-  Developed  hybrid  (analogical/propositional)  representational  framework 
for  planning  avoiding  classic  limitations 

-  Observed  2  to  90  times  speed  up  solving  classic  planning  problems  such 
as  Blocks  World  and  Eight-Square 

•  Glasgow  &  Malton  2003 

-  Applied  array  theory  to  capture  spatial  relations  as  symbolic  arrays 

-  Observed  computational  benefit  from  the  implicit  representation  of  spatial 
and  topological  constraints 

•  Other  work 

-  Barwise  &  Etchemendy  1995 

-  Fobus  &  Gentner  (Davies)  theory  of  rerepresentation  to  generate 
suggestions  based  on  a  library  of  general  methods,  with  strategies  for 
controlling  the  process 
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APPENDIX  D:  DARPA  ENDSTATE  EBUNT  BRIEFING 


6  New  England  Executive  Park  Burlington,  MA  01803 
3811  N.  Fairfax  Drive  Arlington,  VA  22203 
4445  Eastgate  Mall  San  Diego,  CA  92121 


ALPHATECH,  Inc. 


781.273.3388 

703.524.6263 

858.812.7874 


EBUNT  Briefing 


Project  Manager:  Eric  Jones 
Project  Lead:  John  Everett 
August,  2003 


4\. 


Cities  are  the  New  Battlefield 


ALPHATECH,  Inc. 


“The  future  of  warfare  lies  in  the  streets,  sewers,  high-rise  buildings,  industrial  parks,  and 
sprawl  of  houses,  shacks,  and  shelters  that  form  the  broken  cities  of  our  world... " 


•  Successful  employment  of  force 
requires  a  detailed  understanding  of 
a  complex  and  rapidly  evolving 
environment 

-  Buildings  -  Infrastructure 

-  People  -  Resources 

•  Current  urban  ops  planning  tools 

-  Lack  high-fidelity,  coupled  models  of 
indirect  effects 

-  Do  not  incorporate  ISR  tasking  to 
update  or  deconflict  model  parameters 

-  Lack  indicators  for  alternative  opponent 
courses  of  action 

-  Do  not  consider  3-D  urban  environment 

•  Goal:  Rapidly  develop,  update,  and 
exploit  dynamic  models  of  centers  of 
gravity  (COGs)  for  effects-based 
tasking  order  (ETO)  development  in 
urbanized  terrain 


Ralph  Peters,  Fighting  for  the  Future:  Will  America  Triumph ? 


6  New  England  Executive  Park  Burlington,  MA  01803  781-273-3388  3811  North  Fairfax  Drive,  Arlington,  VA  22203  703-524-6263  4445  Eastgate  Mall,  San  Diego,  CA  92121 858-812-7874 
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Problem:  Attrition-Based  engagements  in  Urban  Ops 


Today’s  engagement  planning  support  tools  were  not  developed  for  urban  warfare 
and  produce  attrition  based  COAs....and  have  the  potential  to  produce 
a  multitude  of  negative  effects  &  ROE  contradictions 


6  New  England  Executive  Park  Burlington,  MA  01803  781-273-3388  3811  North  Fairfax  Drive,  Arlington,  VA  22203  703-524-6263  4445  Eastgate  Mall,  San  Diego,  CA  92121  858-812-7874 


Three  Technology  Opportunities  for  Urban 
Operations 


Situation  Report  Fusion 


Maneuver  and  Targeting  CoA  Analysis 
Urban  Effects-Based  Target  Development 


6  New  England  Executive  Park  Burlington,  MA  01803  781-273-3388  3811  North  Fairfax  Drive,  Arlington,  VA  22203  703-524-6263  4445  Eastgate  Mall,  San  Diego,  CA  92121 858-812-7874 


57 


Situation  Report  Fusion 


Distributed,  mobile 
sensor/comms  net 

Form  World  State 
Hypotheses: 

-  Friend  and  Foe 
locations 

-  Structural  damage 

-  Obstacles 

Comms 

Management 

-  Re-establish  contact 
with  unit,  sensors 

Possible  Worlds 
Visualization 


Gathering 
crowd,  from- 
imagery 


S&V 


Most  likely 
path  for  E 


/ 


Estimates  from 
simulation  run 
during  loss  of 
communications 


Iff 


2  minute  possible 
location  ring 


■  V1 

M 


□ 

<tj  #4-| - -  Objective 

]  □  r 


□ 


€> 

Enemy 


TL7  □□  C 


Green  in  location  ring  indicates  Yellow  is  estimated 
possible  positions  for  E  enemy  field  of  fire 
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4.  Information  Requirements  for  Urban  Operations 


•  Mobility 

-  What  are  all  the  routes  across  this  river  within  the  inner  city? 

-  What  are  our  viable  ingress/egress  options  from  this  neighborhood? 

-  Have  we  isolated  this  building? 

•  Model  construction  and  querying 

-  Build  a  network  model  of  the  electric  power  grid 

-  Find  all  the  power  plants  in  this  city 

-  Identify  all  the  buildings  in  this  neighborhood 

•  Tactical  targeting  options 

-  How  can  we  cut  electrical  power  to  this  city  block? 

•  Activity  and  threat  identification 

-  Notify  me  of  changes  in  the  environment  that  could  impact  my  plans 

-  Where  might  the  friendly  squad  be  that  we  lost  contact  with  three  minutes  ago? 

-  What  about  the  enemy  platoon  that  we  observed  five  minutes  ago? 

•  Plan  critiquing 

-  Can  I  flood  this  tunnel?  Will  flooding  it  achieve  my  goal? 


www.  alphatech.  com 
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4S.  Complicating  Factors 


ALPHATECH,  Inc. 


•  Answers  to  these  questions  often  change  over  time 

-  Snipers  block  best  egress  route 

-  Enemy  activity  in  building  may  indicate  subterranean  access  routes 

-  Newly  rubbled  building  impedes  access  to  electricity  substation 

-  New  observation  of  enemy  force  location  updates  predictive  estimate 

•  Answers  to  these  questions  are  often  uncertain 

-  Based  upon  fragmentary  data  from  multiple  sources 

•  Answers  are  buried  in  a  glut  of  information 

-  Rapid  change  requires  early  recognition  of  emerging  scenarios,  threats 

-  First  indications  of  change  may  appear  anywhere  in  the  modeled  area 
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Limitations  of  Conventional  GIS  Systems 


ALPHATECH,  Inc. 


•  Provide  inadequate  fidelity  for  urban  operations 

-  Often  outdated  and  inaccurate 

-  Wrong  scale  and  lacking  in  detail 

-  Two-dimensional 

-  Static 

•  Do  not  provide  necessary  support  for  model  update 

-  Cannot  represent  uncertainty  regarding  model  structure 

-  Cannot  represent  evidence  pedigrees 

-  Conventional  vector  GIS  schemas  (e.g.  VPF)  base  representations  on  lines, 
points,  and  regions,  and  have  no  notion  of  object/relational  structures 

•  Have  no  representation  of  capabilities,  plans  and  goals 

-  Warfighters  must  superimpose  their  knowledge  of  plans  and  objectives  on  the 
information 
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Cs-  Proposed  New  Capability 


ALPHATECH,  Inc. 


High-fidelity  urban  model 

-  Registered  geospatially 

-Assembled  from  multiple  information  sources 

-  Encompasses  3D  representations  of  spatial  features 

-  Evolves  over  time  in  response  to  intelligence 

-  Supports  a  uniform  probabilistic  framework 

-  Assimilates  new  information 

-  Maintains  pedigree  for  fusion  and  visualization 

-  Contains  multiple  layers  of  information 


www.  alphatech.  com 


Gs- 


Layers  of  Representation 


ALPHATECH,  Inc. 


|>-'^catioiV|cor^ample^^at^ 


COA  layers 


Battlefield  Effects 


Friendly,  Adversary,  Civilian  locations 


Network  infrastructure  layers 


3D  object 


Draped  3D  wireframe 


Vector  G  IS 


•  Layers  may  be  partially  derived  from 
other  layers 

-  Infrastructure  layer  from  GIS  data, 
imagery 

•  Queries  can  access  an  arbitrary 
subset  of  layers 

-  Probabilistic  framework  fuses 
information  across  layers 

•  Each  layer  can  have  temporal 
extent 
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Cs-  Proposed  New  Capability 


ALPHATECH,  Inc. 


Model  V 

Model 

Construction  |  ^ 

Exploitation 

Tools 

Tools 

Data  fusion  and  knowledge  based  tools  for  rapid  model  development 
and  updating 

-  Legacy  model  import 

-  Georegistration  for  multiple  spatially-distributed  data  sets 

-  Vector  GIS  products 

-  Blueprints 

-  Imagery 

-  Multi-source  fusion  for  automated  model  refinement/update 

-  Sketch-based  operator  input 

-  Ontology  and  probabilistic  rule  base  for  inferring  most  likely  structure 

-  Rapid  knowledge  formation  support  for  extending  knowledge  base 

-  Supervised  and  unsupervised  machine  learning 
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Gs-  Proposed  New  Capability 


ALPHATECH,  Inc. 


Model 

Exploitation 

Tools 


Tools  for  Rapid  Situation  Understanding 

-  Situation  monitoring  ( filter  information,  post  alerts  for  users) 

-  COA  evolution  monitoring 

-  Monitor  force  positions  relative  to  objectives 

-  Salient  change  detection 

-  Explanation-based  alerting 

-  Activity  identification 

-  Enemy,  civilian  location 

-  Decision  support  ( query-driven ,  user-responsive) 

-  Network  reasoning 

-  Mobility  analysis 

-  Infrastructure  targeting 

-  Plan  critiquing 


www.  alphatech.  com 


12 


61 


€&—  What’s  New?  Why  Now? 


ALPHATECH,  Inc. 


•  First-ever  large-scale  geospatial  knowledge  base  to  incorporate 
explicit  representations  of  time,  uncertainty,  plans,  and  goals 

•  Fundamentally  new  approach  to  developing,  representing,  and 
exploiting  military  geospatial  information 

-  Grounded  in  entity-centric  representations,  not  points,  lines,  and  regions 

-  Designed  for  exploitation  and  model  update 

-  Integrates  sensor-derived  data  with  plan  and  goal  representations 

•  Many  of  the  necessary  enabling  technologies  are  in  place 

-  Scalable,  object-oriented  GISs 

-  Fusion  for  3D  model  creation 

-  Knowledge  entry  tools 

-  Knowledge  representation 

-  Probabilistic  (Bayesian)  formalisms  for  relational  representations 

-  Example-driven  machine  learning  from  imagery 

-  Services-based  architectures 


www.  alphatech.  com 
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ENDSTATE  Concept :  Situation  Report  Fusion 


ALPHATECH,  Inc. 
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€&—  Realizing  the  ENDSTATE  Vision 


ALPHATECH,  Inc. 


•  COTS  and  maturing  research  systems  provide  the  basis 

-  GIS  databases 

-  3D  visualization 

-  CAD  model  construction  from  imagery 

-  Supervised  machine  learning 

•  Advances  in  data  fusion  and  knowledge-intensive  methods  are 
essential  for  realizing  radical  new  capabilities 

-  Overcoming  noisy  or  missing  input  data  during  structure  modeling 

-  Identifying  key  structural  features  of  the  terrain  (e.g.  major  intersections) 

-  Monitoring  model  evolution  over  time  relative  to  plans  and  goals 

-  Providing  plan  critiques  and  COA  suggestions  that  combine  the 
environment  model  with  plans  and  goals 
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ENDSTATE:  Inference  Across  Database  Layers 


ALPHATECH,  Inc. 


Using  MTI  track  data  to  fill  in  gaps  and  correct 
errors  in  GIS  road  network  information 


Mitrovitsa 


Kosovo  Road  network 


www.  alphatech.  com 
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ENDSTATE:  Identifying  Key  Landscape  Features 


ALPHATECH,  Inc. 


Major  mobility 
corridors 


Close-block 
orderly  urban 
area 


Several  possible  approaches 

•  Supervised  machine  learning 

•  Extract  from  GIS  data 

•  Utilize  urban  area  knowledge 

•  Example:  Factories  are  unlikely 
in  residential  areas 


Open  areas 
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ENDSTATE:  Situation  Monitoring 


ALPHATECH,  Inc. 
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4S.  Situation  Report  Fusion  Summary 


ALPHATECH,  Inc. 


•  Information  technology  is  on  the  brink  of  enabling  radically  new 
GIS-based  situation  understanding  systems 

•  Challenging  research  efforts  are  required  in 

-  Data  fusion 

-  Image  understanding 

-  Large-scale  knowledge-based  inference 

•  Payoff  will  be  systems  that 

-  Develop  high-fidelity  3D  models  of  geographic  areas 

-  Monitor  unfolding  events  within  these  areas 

-  React  effectively  to  changes  that  may  impact  plans  and  goals 


www.  alphatech.  com 
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Maneuver  and  Targeting  CoA  Analysis 


Critiquing  Courses  of  Action 


Validate  whether  ground 
forces  CoAs  achieve  their 
purposed  effects 

Evaluate  both  maneuver 
and  targeting  CoAs  for 
company  echelon  and 
below 

Suggest  modifications  to 
original  CoA 
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The  Future  of  Urban  Tactics 


•  Common  Operational/Tactical  Picture 

•  DARPASUOSAS 

•  C2  of  Combined  Arms  Teams  down  to  squad  level 

•  Netted  Fires 

•  UGV/UAV  logistics,  sensor,  and  fires  platforms 

•  Enhanced  mobility,  maneuver  warfare  tactics 
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Example:  New  Marine  Urban  Combined  Arms 
4V  CONOPS 


ALPHATECH,  Inc. 


•  Urban  Combined  Arms  Company 

-  1  Inf  company 

-  1  Tank  platoon 

-  1  AAV  platoon 

-  1  LAV  platoon 

-  Organic  mortar  and  anti-tank  capability 

-  Eschew  air  support  and  long-range  arty 

-  Direct  over  indirect  fire  support 

•  Squad  leader  controls  2  fire  teams  and  1  tank  team 
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ENDSTATE  Concept:  Small  Unit  Effects- 
Based  Command  and  Control 
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ENDSTATE:  Plan  Critiquing 


Enemy  contacts  have 
been  reported  around  this 
building,  in  the  yellow  area 
thought  to  be  secured 


Potential  plan 
to  secure 
area:  Use  river 
to  flood  tunnel 


□ 
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Red  force  Blue  force 
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Maneuver  CoA  Analysis  Summary 


•  Evolving  CONOPS  for  urban  tactics  create  challenges  for 
effective  tactical  C2 

-  Control  of  combined  arms  and  netted  fires  at  smaller  echelons 

-  Use  of  unmanned  vehicles  for  recon  and  force  projection 

•  Challenging  research  efforts  are  required  in 

-  Scalable  contingency  planning  in  complex  urban  environments 

-  Large-scale,  detailed  knowledge  representation  of  urban  landscape 

-  Hands-free,  heads-up  displays  for  plan  authoring  and  critiquing 

•  Payoff  will  be  systems  that 

-  Provide  tactical  suggestions  for  the  use  of  small  combined  arms 
teams  as  change  occurs 

-  Alert  ground  commanders  to  implications  of  tactical  decisions 
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Urban  Effects-Based  Target  Development 


•  Develop  ground-based,  small-unit  courses  of  action 

-  Use  models  of  environment,  infrastructure,  and  adversary 

-  Predict  and  assess  which  lethal  and  non-lethal  actions  produce 
desired  effects 
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What  is  Effects-Based  Urban  Network 
a.  Targeting? 


ALPHATECH,  Inc. 


•  Effects-based  operations  (EBO)  concept... 

-  Select  operational  tasks  that  achieve  desired  effects  and  reduce/eliminate 
undesired  effects 

-  Account  for  effects  by  understanding  mechanisms 

-  Use  full  range  of  elements  of  national  power 

•  DIME  (diplomatic,  information,  military,  economic) 

•  ...applied  to  urban  warfare. 

-  Model  city  plus  inhabitants  as  complex  adaptive  system  of  systems  to  find  centers 
of  gravity  (COGs) 

•  PMESI2  (political,  military,  economic,  social,  infrastructure,  information) 

-  Exploit  IPB/ISR  to  populate/update  COG  target  set  analysis  (TSA)  models 

-  Develop  targets,  fires  and  maneuver  effects,  and  ROEs  for  the  urban  fight 
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Emerging  Effects-Based  Urban  Doctrine 


EBUNT  plans  Maneuver  Warfare  effects  in  addition  to  Precision  Fires  effects 
Concept  of  Critical  Nodal  Assault 


Deep  Strike 

some  nodes  can  be 
destroyed  by  indirect 
precision  fires 


Close  Combat 

some  nodes  can  be 
seized  by  small  unit 
precision  maneuver 
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Why  apply  EBO  to  urban  operations? 


•  Achieve  economy  of  force 


•  Avoid  attrition  warfare  or  siege 

•  Preserve  legitimacy  and  reduce  noncombatant  casualties 

•  Support  full  spectrum  operations,  including  MOOTW 
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Problem:  Urban-specific  challenges  for  EBO 


Urban  terrain  exacerbates  imperfect  information 
Greater  emphasis  on  human  factors  COGs  (political,  economic, 
and  social)  relative  to  terrain  and  buildings 

Greater  density  of  potentially  undesirable  effects 

-  Collateral  damage,  negative  media  impact,  tightly  coupled  COGs 


Urban  terrain  fragments  command  and  control 

•  Requires  rapid  decision  making  at  tactical  level  with  operational 
significance 

-  Chasing  bad  guys  through  neighborhood  X  may  endanger  operational  goal  of 
cordial  relations  with  native  ethnic  group 

•  Targets  are  dispersed,  hidden,  mobile,  or  politically  sensitive 
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Emerging  Technologies  for  EBUNT 


Technical  Challenges 

Promising  New 
Technologies 

Enhanced  Operational  Net  Assessment 

Rapid  Knowledge  Formation  for  building 
COG  models 

Increased  importance  of  human  factors 
in  urban  COG  Analysis 

Belief  networks  to  support  wargaming, 
esp.  leadership  and  sociopolitical 
models 

Increased  infrastructure  density  and 
detail,  tighter  coupling  of  COG  networks 

Hybrid  optimization  and  inference 
techniques  for  high-fidelity  target  set 
analysis 
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Operational  Scenario  1: 

Initial  Urban  Model  of  Militacity  Retrieved  By  Reach-back 
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Operational  Scenario  2: 

Intel  Officer  Fills  Model  With  Current  ISR 


Commander’s  Intent:  Destroy  militia  command 
outpost  in  Militacity 

Rules  of  Engagement :  Protect  civilians  and  urban 
infrastructure 


Combined  Operations  Intel  Center.  Outpost  located 
in  bottom  left  sector  of  city  of  10,000 
No  other  known  red  targets 


Intel  Officer  Fills  Model 
With  Current  ISR 
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Operational  Scenario  3: 
Initial  EBUNT  Development 


New  Goal:  Repair 
Increase  Civil  Support 


Revised: 

•Dismount  Attack  vs.  Precision  Fin 
•Tactically  Local  Attack  on 
EP  &  LOC  utility  connections 
•Update  ISR 


New  Goals: 

•Provide  Protection 
•Increase  Civil  Support 


New  Goals: 

•Distribute  PSYOPS  Leaflets 
•Collect  New  Intel 
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Operational  Scenario  4: 

New  Intel  From  Civilians  Entered  Into  Model 
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Operational  Scenario  5: 
Update  EBUNT 


New  Goal: 

•  Direct  Fire 

•  Dismount  Attack  on  Camp 


Intel  Officer  Fills  Model 
With  New  ISR 


New  Goal:  Repair 
Increase  Civil  Support 


New  Goal: 

Protect  Apartment  Bldg 
Tactically  Local  Attack  on  Supplies 
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Example  EBUNT  Program  Component: 
Enhancing  Operational  Net  Assessment 


•  Intel  officers  must  rapidly  populate  COG  models 

-  72-96  hour  deployment  lead  time  for  some  urban  ops 

-  Need  consistent  interpretation  of  infrastructure  data  from  world-wide 
repositories 

•  DAML+OIL  ontology  markup  language  provides  semantics 

•  Key  ontology  elements 

-  Represent  EP+  POL  infrastructure 

-  Represent  supported  activities  and  interdependencies 

•  Leverage  MIDB  schema 

-  Many  ontology  elements  used  in  COG  analysis  already  captured 
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EBUNT  ONA  Technology  Concept 


DAML  +  OIL  infrastructure  data  sources 


Model 
Visualization 


> . . 


1  ! 


cLBj  Infrastructure 

ontology 
components 


)H»iW 


structure 


Knowledge 

Acquisition 

Toolkit 


1.  Query  DAML  Infrastructure  Data 
Sources  using  DAML  Agent 

“What  sorts  of  data  are  available  concerning 
Jihadia’s  oil  pipeline  pump  stations?” 


Extraction 


Infrastructure 

Model 


Infrastructure  model 
templates 


6  New  England  Executive  Park  Burlington,  MA  01803  781-273-3388  3811  North  Fairfax  Drive,  Arlington,  VA  22203  703-524-6263  4445  Eastgate  Mall,  San  Diego,  CA  92121 858-812-7874 


EBUNT  ONA  Technology  Concept 


DAML  +  OIL  infrastructure  data  sources 
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COG  Analysis 


2.  Build  modeling  templates  using 
rapid  knowledge  acquisition  tools 

“To  run  my  POL  analysis  application,  I  need  data  for 
these  features  of  pump  stations  and  pipeline  joints” 


Infrastructure  model 
templates 
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EBUNT  ONA  Technology  Concept 


DAML  +  OIL  infrastructure  data  sources 


Model 
Visualization 
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3.  Populate  model  templates  using 
instances  from  DAML  sources. 

“Sources  include  M&S  relevant  data  for  twenty 
Jihadia  pump  stations.” 


templates 
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4.  Run  COG  analysis,  visualize  1 
models  and  analysis  results. 

“My  POL  analysis  application  suggests  this  optimal 
Jihadia  CoA  if  we  target  that  station." 
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Infrastructure  model 
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Urban  Network  Targeting  Summary 


•  EBUNT  is  a  challenging  extension  of  effects-based 
planning  to  the  urban  environment 

-  Intel  collection  and  exploitation 

-  Sociopolitical  modeling 

-  Detailed,  complex,  tightly-coupled  urban  COGs 

•  EBUNT  provides  target  development,  fires/maneuver 
effects  assessment  and  planning 

-  Analogous  to  target  development  and  weaponeering  in  air  ops  cycle 
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EBUNT  technology  development 
opportunities? 


•  Are  services  interested  in  a  technology  solution  to  this 
problem? 

-  Army/Marine  commanders  are  trained  to  select  objectives/targets 
that  achieve  desired  effects 

•  Skepticism  about  performing  this  task  better  with  automation 

-  Command  structure  for  Army/Marines  is  necessarily  decentralized 

•  Different  model  than  Air  Force  central  planning/decentralized 
execution 

-  Real  need  is  for  more  complete  intelligence  picture  and  analysis 

•  More  HUMINT  and  better  IPB  on  the  urban  environment 
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APPENDIX  E:  DARPA  ENDSTATE  FOGLIGHT  FINAL  SUMMARY  BRIEFING 


Fog  Light 

Dynamic  Interfaces  for  the  Warfighter 

"The  great  uncertainty  of  all  data  in  war  is  a  peculiar  difficulty,  because  all  action  must,  to  a 
certain  extent,  be  planned  in  a  mere  twilight,  which  in  addition  not  unfrequently  —  like  the  effect  of 
a  fog  or  moonshine  —  gives  to  things  exaggerated  dimensions  and  unnatural  appearance.11 

Clausewitz 


30  November,  2004 


Battlefield  Stress  Makes  Interfaces  Ineffective 


•  The  platoon  soldier  is  an  18  year  old  recruit  from  Arkansas 

-  Not  seasoned 

-  Not  a  lot  of  education 

-  Basic  training  only 

•  Command  direction  must  be  clear  and  crisp  in  all  circumstances 

-  The  “fog  of  war”  impedes  decision  making  in  the  middle  of  battle 

-  Force  fragmentation  in  urban  combat  forces  decision  making 
down  the  chain  of  command 

•  Current  and  next  generation  C2  systems  fail  to  improve  prosecution 
while  moving  the  next  10  feet 

-  No  time  to  interpret  a  fire  hose  of  information 

-  Lack  skill  interpreting  complex  map  symbology 

-  Mental  stress  impacts  reasoning  from  maps  to  real  scene 
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What  are  We  Trying  to  Do? 


The  warfighter  needs  computer  interfaces  that  adapt  to  the  context  and  the  individual 

•  Vision 

-  Decision  support  tools  that  customize  themselves  to  the  user  based  on 
understanding  of  information  displayed,  individual,  and  situation 

-  Effective  user  interfaces  for  dynamically  composed  command  direction 

•  Approach 

-  Combine  diagrammatic  reasoning  techniques  and  intelligent  interface  methods 
drawing  on  psychophysical  understanding  of  human  sensing  systems 

-  Develop  context-aware  systems  that  can  transmit  information  in  a  best-first  order 

•  Benefits 

-  More  accurate  decisions  under  duress 

-  Improved  utilization  of  next  generation  C2  systems  (e.g.,  HURT,  RAID) 

-  Reduced  development  time  for  more  effective  interfaces 


Battlefield  Stress  Makes  Interfaces  Ineffective 


•  The  platoon  soldier  is  an  18  year  old  recruit  from  Arkansas 

-  Not  seasoned 

-  Not  a  lot  of  education 

-  Basic  training  only 

•  Command  direction  must  be  clear  and  crisp  in  all  circumstances 

-  The  “fog  of  war’’  impedes  decision  making  in  the  middle  of  battle 

-  Force  fragmentation  in  urban  combat  forces  decision  making 
down  the  chain  of  command 

•  Current  and  next  generation  C2  systems  fail  to  improve  prosecution 
while  moving  the  next  10  feet 

-  No  time  to  interpret  a  fire  hose  of  information 

-  Lack  skill  interpreting  complex  map  symbology 

-  Mental  stress  impacts  reasoning  from  maps  to  real  scene 
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What  are  We  Trying  to  Do? 


The  warfighter  needs  computer  interfaces  that  adapt  to  the  context  and  the  individual 

•  Vision 

-  Decision  support  tools  that  customize  themselves  to  the  user  based  on 
understanding  of  information  displayed,  individual,  and  situation 

-  Effective  user  interfaces  for  dynamically  composed  command  direction 

•  Approach 

-  Combine  diagrammatic  reasoning  techniques  and  intelligent  interface  methods 
drawing  on  psychophysical  understanding  of  human  sensing  systems 

-  Develop  context-aware  systems  that  can  transmit  information  in  a  best-first  order 

•  Benefits 

-  More  accurate  decisions  under  duress 

-  Improved  utilization  of  next  generation  C2  systems  (e.g.,  HURT,  RAID) 

-  Reduced  development  time  for  more  effective  interfaces 


Context-Aware  Information  Systems 


•  Conventional  interfaces  are  minimally  aware  of  device 

-  Screen  size,  color  depth 

•  Context-aware  systems  must 

-  Take  into  account  user  context 

-  User  preferences  and  cognitive  style 

-  Interactions  with  other  users 

-  Current  task  and  situation 

-  Determine  interface  parameters  that 

-  Encode  critical  elements  of  the  current  problem  and  its  solution 

-  Generate  effective  interfaces  on  the  fly 
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Example:  Reacting  to  a  Flanking  Attack 


Enemy  company 


Useful  Display  Transformations 

1.  Derive  decision  space  from  user  role 

2.  Highlight  salient  graphical  elements  (e.g., 
assets,  control  measures) 

3.  Gray  out  irrelevant  elements  of  display 

4.  Highlight  implications  of  decisions 

5.  Transform  distances  into  directly 
comparable  graphical  elements  (e.g.,  pie 
charts  showing  time  to  target) 

6.  Transform  other  relevant  dimensions  (e.g., 
unit  strength)  into  common  graphical 
elements  (e.g.,  color  of  time-to-target 
slice). 


Role-Specific  Interfaces 


•  User’s  characteristics 

-  Role 

-  Prefers  tabular  data 

•  Device 

-  Size  of  display 

-  Touch-screen  capability 

•  Reasoning  context 

-  Interrupfibility 

•  Data  sources 

-  Speed  of  comms  link 

-  Quality,  quantity  of  data 
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Computing  What  Data  to  Display 


0  10  20  30  40  50  60  70 


Presentation 


Friendly  forces  at  <40,20> 

1  tank  company,  40%  fuel  level,  range  60  miles 
Weapons  range  2  miles 

Enemy  forces  at  <30,60> 

1  tank  company 
Weapons  range  1.5  miles 

Lake  at  {<10,20>,<21,41>,<35,50>,<48,45>,<54,61>, 
<60,60>,<58,49>,<52,32>,<40,37>,<35,27>, 
<23,27>,<20,17>,<15,15>} 


Display  directly  represents  key  inferences 

•  Lake  is  a  mobility  barrier 

•  Lake  is  not  a  barrier  to  fires 

•  Units  are  not  in  range  for  engagement 

•  Blue  force  can  reach  red  force  with  current 
fuel  reserves 

•  Display  of  engagement  range  would  change  if 
temperature  were  below  freezing,  enabling 
one  or  both  forces  to  traverse  lake 

•  If  lake  were  a  mountain,  fires  boundary  would 
not  cross  line 


Air  temperature  40°F 


Challenge:  Determine  Principles  for 
Mapping  Situation  Data  to  Interfaces 


Situation  Data 
00110001010101 
10100101100111 
01101011101010 


Data  describing  situation 
underdetermines  structure 


Determines  choice  among 
otherwise  valid  structures, 
based  on  current  context 


/ 


FOGLIGHT 
Dynamic 
Interface  - 
Generator 

Research  Goal: 
Develop  general 
principles  for  mapping 
from  situation  data  to 
useful  interfaces 


Findings  from  cognitive  research 
on  human  perception 


User  Interface 


Representation 
of  information 


Recent  advances  in  logic  for  describing 
mappings  between  semiotic  systems 


Interfaces  require  commitment 
to  a  particular  structure 
Interface:  windows,  menus, 
dialog  boxes,  etc. 

Information:  tables,  bar  charts, 
trend  graphs,  etc. 

Conventional  interfaces 


Provides  a  basis  for 
constructing  a  general, 
provably  correct  system  of 
information  transformations 
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Pre-Attentive  Visual  Features 


Feature 

Researchers 

Line  (blob)  orientation 

Julesz  &  Bergen  (1983);  Wolfe(1992) 

Length 

Triesman  &  Gormican  (1988) 

Width 

Julesz  (1985) 

Size 

Triesman  &  Gelade  (1980) 

Curvature 

Triesman  &  Gormican  (1988) 

Number 

Julesz  (1985) 

Terminators 

Julesz  &  Bergen  (1983) 

Intersection 

Julesz  &  Bergen  (1983) 

Closure 

Enns  (1986);  Triesman  &  Souther  (1985) 

Color  (hue) 

Triesman  &  Gormican  (1988);  Nagy  &  Sanchez  (1990) 

Intensity 

Triesman  &  Gormican  (1988);  Beck  et  al.  (1983);  Julesz  (1971) 

Flicker 

Julesz  (1971) 

Direction  of  motion 

Nakayama  &  Silverman  (1986);  Driver  &  McLeod  (1992) 

Stereoscopic  depth 

Nakayama  &  Silverman  (1986) 

3D  depth  cues 

Enns  (1990) 

Lighting  direction 

Enns  (1990) 

85 


Display  for  UA  XO 


•User 

-  Skilled  at  map  interpretation  / 

-  Capable  of  fast  accurate  recall 

-  High  training  level 


•Role 

-  FCS  XO 

-  Decision  maker 

-  Task:  create  counter-attack  plan 
without  compromising  mission 
objective,  time  and  resource 
constrained 

•Device  / — 

-  FCS  command  vehicle  ' 

-  Workstation  with  good  comms 


Use  conventional  symbology 
on  a  map-based  interface 


Sacrifice  low-level 
details  in  order  to  show 
lig-picture  context 


Graphically  show 
critical  deadlines 
for  individual  units 


Make  use  of  graphical 
features  that  require 
high  resolution 


•  KB 


■  COP  including  enemy  and  friendly- 
locations,  with  missing  data 


•Reasoning  Mode 

-  Orient/Decide 


_Refresh  when  new 
data  enters  COP 


Keep  display  relatively  fixed,  graying 
out  irrelevant  aspects,  while 
preserving  context  to  support  recall 


Display  for  Platoon  Leader 


•KB  Adjust  update 

-  COP  including  enemy  and  —  rate  t0  ensure 
friendly  locations,  with  accurate  picture 

missing  data 


•User 

-  Skilled  motor  control  / 

-  Weakly  learned  symbology 

-  Low  training  level 

•Role  / 

-  Platoon  leader 

-  Coordinate  recon  effort " 
under  deadline 

•Device 

-  Portable  workstation 

-  Remote  sensors 


Employ  a  3D,  overview 
perspective  display, 
avoid  symbolically  rich 
map-based  displays 


Provide  an  overview  of 
immediate  area  of  action 

_  Make  member  status 
evident  in  the  display 


Display  all  platoon  mem 
and  any  known  enemies,  in 
the  context  of  trafficability 
constraints 


•Reasoning  Mode 

-  Orient/decide 


Minimize  display 
reconfiguration 
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Display  for  Platoon  Soldier 


•User  / 

-  Skilled  motor  control  ^ 

-  Weakly  learned  symbology 

-  Low  training  level 

•Role  / 

-  Platoon  soldier 

-  Recon  task:  time 


Employ  a  3D,  first  person  perspective 
display,  avoid  symbolically  rich,  map- 
based  display 


Provide  a  localized,  highly 
detailed  point  of  view 

_  Reify  task  progress, 
deadlines,  direction 


constrained,  identify  type  & 
number  of  enemy  Display  all  known  adversaries 

^ — and  any  known  cover  and 
trafficability  constraints 


•Device 

-  Heads-up  display 

-  Remote  sensors 


Minimize  input 
requirements 


•  KB 

-  COP  including  enemy  and-  enTufeTccumtepicture 
friendly  locations,  with  ensure  accurate  plcture 
missing  data 


_  _ Allow  display  to  reconfigure  to 

*  Reasoning  Mode  bring  new  threats  into  focus 


-  Observe/orient 


How  is  it  Done  Today? 


•  Minimal  commitment,  maximum  generality 

-  Microsoft  Word  displays  any  text,  Microsoft  Excel  displays  tabular  data 

-  No  commitment  to  semantics  of  information 

-  Major  engineering  effort  to  make  interface  useful 

-  No  consideration  given  to  user’s  environment  or  task  context 

•  Mission-specific  interfaces 

-  Developed  for  particular  tasks  (e.g.,  Command  Post  of  the  Future) 

-  Single-purpose,  designed  with  respect  to  semantics  of  information 

-  Minimal  consideration  given  to  user’s  environment  or  task  context 

•  Accommodating  user  variance  causes  unmanageable 
requirements 

•  Cognitive  task  analysis  can  create  user  models,  but  is  labor 
intensive 

•  Recent  work  on  generating  interfaces  from  parametric 
descriptions  focuses  on  alternative  displays 

-  Web  phones 
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Generating  Interfaces  from  Parameters 


•  Leverage  research  on  context-aware  Ul  design:  mapping  interface 
parameters  to  a  Ul  based  on  rules. 

•  Generally  a  three-step  process: 

-  Framework  for  designing  rules  (how  do  we  map  IP  to  Ul?) 

-  No  general  solution;  design  of  rules  is  problem  dependent 

-  Project  goal:  framework  for  rule  design  and  experimental  methodology  for 
instantiation 

-  Framework  for  encoding  rules  (how  are  rules  stored  and  implemented?) 

-  E.g.,  TERESA  (Mori,  Paterno,  Santoro) 

-  Framework  for  runtime  support  (how  do  rules  generate  multiple  Uls  at  runtime?) 

-  E.g.,  Dygimes  (Clerckx,  Lyuten,  Coninx) 


Example  for  Encoding  Rules:  TERESA 


Given  task  model, 
create  framework 
for  multiple  Uls 


- D>>  —  i  — [>  —  Iff 

Select_artwork  Show_artwork_info*  Closejnfo 


Vatteroni's  Donation 

Title:Figure 

Artist:  Vatteroni  Felice 

Material:  Pink  Marble 

Options  Back 
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What’s  New,  and  why  will  it  Work? 

•  Enabling  the  principled  composition  of  user  interface  elements  to  construct 
interfaces  that  preserve  key  structural  properties  of  the  input  data 

-  Informed  by  cognitive  perceptual  theory 

-  Grounded  in  a  sound  logical  basis  that  provides  the  means  for  verifying  soundness  and  proving 
properties  of  the  system 

•  Combining  the  advantages  of  current  paper-based  planning  tools,  such  as  maps  and 
sketches,  with  automated  inference  processes  that  dynamically  generate  interface 
displays 

-  Configuring  displays  on  the  fly  based  on  user’s  context 

•  Achieving  a  lOx  improvement  in  performance 

-  Reduce  training  time  (direct  encoding  and  task-based  user  interface  generation  will  lower 
learning  curves) 

-  Reduce  time  on  task  (compressing  inference  steps  and  reducing  user  interactions  with  the 
system  because  the  interface  displays  elements  of  the  structure  of  the  solution) 


Who  Cares?  What  Difference  will  it  Make? 

•  Enable  Army  FCS  to  achieve  its  Warfighter  Machine  Interface  (WMI) 
objective 

-  Enable  machine  interfaces  to  adapt  to  human  capabilities 

•  Improve  communications  in  joint  operations 

•  Reduce  need  for  staff  specially  trained  in  single-purpose  displays 

-  Develop  principles  of  interface  synthesis  that  transcend  particular  applications 

•  Improve  situation  awareness  and  decision  quality  in  dynamic 
replanning 

-  Configure  interfaces  to  support  human  decision  making  under  stressful  conditions 

•  Enable  faster  task  completion  with  less  training 

-  Get  inside  the  enemy’s  decision  loop 
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Diagrammatic  Reasoning  For  Joint  Operations 


Diagrammatic 

Representations 

containing  graphical 
glyphs  and/or  images 
using  graphical 
abstractions  (spatial, 
temporal,  geometric, 
or  topological) 


Propositional 

Representations 

containing 

predicates,  symbols, 


Algebraic  theory 
capturing  analogical 
representations 


Info  preserving 
mapping  of 
spatial,  geom,  & 
temporal 
relationships 


Air  Force 
Qualitative 
Plan 
Trace 


Army 

Qualitative 

Plan 

Trace 


Research  Pointing  The  Wav 

•  Goguen  &  Harrell  2003 

•  Garagnani  &  Ding  2003 

•  Barwise  &  Etchemendy  1995 

•  Glasgow 


What  are  the  Risks  and  the  Payoffs? 

•  Risks 

-  Scientific  foundation  not  yet  sufficiently  established 

-  Computers  not  yet  fast  enough 

•  Payoffs 

-  Shorten  development  timelines  by  raising  the  level  at  which  GUIs  are  specified 

-  Lower  the  barriers  to  adoption  of  automated  C2  systems 

-  Reduce  training  costs  and  staff  requirements  for  new  systems 

-  Interfaces  that  improve  human  cognitive  capabilities  across  all  applications 
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How  Much  will  it  Cost,  over  what  Timeframe? 

•  4  years 

•  $40  million 


What  are  the  Midterm  and  Final  Checks  for  Success? 


•  Best  test  is  evaluation  under  realistic  circumstances 

•  Midterm  evaluation  in  simulated  environment 

-  Focus  on  breadth  of  evaluation 

•  Final  evaluation  in  mixed  simulation/real  wargame  environment 

-  Focus  on  depth  of  evaluation 

•  Extensive  experimentation  framework  an  integral  part  of  program 

-  Conjoint  designs  for  rapid  evaluation  of  multiple  alternatives 

-  Fractional  factorial  designs  for  in-depth  evaluation  of  performance 
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Two-Phased  Experimentation  Framework 


•  Prototype  phase 

-  Multiple  prototype  interfaces  presented  to  various  users  (contexts) 

-  Users  express  preferences  via  ranking 

-  Conjoint  analysis  determines  test  setup  and  provides  optimal  mapping  and 
analysis 

-  Advantage:  inexpensive,  off-line,  no  performance  assessment  required 

-  Disadvantage:  mapping  based  on  user  preferences,  not  performance 

•  Testing  phase 

-  Optimal  interface  and  variants  presented  to  various  users  (contexts) 

-  Assess  user  performance  in  an  exercise 

-  Fractional  factorial  design  determines  test  setup  and  provides  optimal  mapping 
and  analysis 

-  Advantage:  assessment  based  on  performance;  can  correlate  preferences  to 
performance  (determine  limits  on  preferential  adjustability) 

-  Disadvantage:  more  labor-intensive  (battlefield  scenario  required);  requires 
realistic  performance  metrics  (real-valued  AUTL  metrics,  etc.) 


Conjoint  and  Fractional  Factorial 
Design  and  Analysis  of  Experiments 

•  Problem:  Determine  the  “optimal”  configuration  of  an  object, 
where  the  configuration  is  a  combination  of  various  features. 

-  Example:  new  car  design 

-  Features:  color,  horsepower,  style,  gas  mileage,  etc. 

-  Goal:  optimize  features  to  maximize  sales  to  a  given  demographic 

•  Issue:  Can’t  test  all  possibilities  (combinatorial  explosion) 

•  Solution:  Carefully  select  combinations  to  test  so  that  statistical 
inferences  can  be  made  for  any  combination. 

•  Question:  What  is  “optimal”? 

-  Combinations  which  are  preferred 

-  Conjoint  analysis 

-  Combinations  which  perform 

-  Fractional  factorial  design  and  analysis 
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Conjoint  Design  and  Analysis 


•  Determine  optimal  configuration  by  eliciting  user  preferences 

•  Experiment:  Present  carefully  chosen  collection  of  configurations 
to  users  and  solicit  rankings  of  those  configurations  (preferences) 

•  Benefits:  Determine  the  effect  of  various  features  on  preference 

-  Main  effect,  secondary  effects,  etc. 

-  Which  features  are  most  critical,  least  critical,  and  so  on... 

-  Strength  of  effects 

-  Quantify  the  effect  of  a  feature  setting  on  user  rating  or  preference 

-  Quantify  confounding 

-  Interaction  of  various  features 


Fractional  Factorial  Design  and  Analysis 


•  Determine  optimal  configuration  by  assessing  user  performance 

•  Experiment:  Present  carefully  chosen  collection  of  configurations 
to  users  and  assess  (real  valued)  performance  of  those 
configurations  (e.g.,  under  simulated  battle  conditions) 

•  Benefits:  Determine  the  effect  of  various  features  on  performance 

-  Main  effect,  secondary  effects,  etc. 

-  Which  features  are  most  critical,  least  critical,  and  so  on... 

-  Strength  of  effects 

-  Quantify  the  effect  of  a  feature  setting  on  performance 

-  Quantify  confounding 

-  Interaction  of  various  features 
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Why  Now? 


•  Rapid  increase  in  capabilities  and  complexity  of  current  and 
anticipated  C2  systems  will  place  interface  design  on  the  critical 
path 

-  Humans  are  likely  to  remain  in  the  loop 

-  Interface  design  is  a  major  element  of  new  systems  design 

•  Recent  research  has  generated  promising  results 

-  Cognitive  science  results  in  the  study  of  human  perceptual  biases 

-  Emergence  of  alternatives  to  propositional  representations  of  problem  spaces 

-  Advances  in  the  theory  of  algebraic  mappings  between  semiotic  systems 


References 


•  Diagrammatic  Reasoning 

-  Barwise  &  Etchemendy  1995 

-  Developed  theory  of  heterogeneous  reasoning,  incorporating  propositional  and 
diagrammatic  elements 

-  Goguen  &  Harrell  2003 

-  Used  algebraic  abstract  data  type  theory  to  define  semiotic  morphisms 
providing  information  preserving  transformations  between  representations 

-  Applied  theory  of  info  visualization  to  measure  quality  of  several  concrete 
examples,  identifies  general  principles 

-  Garagnani  &  Ding  2003 

-  Developed  hybrid  (analogical/propositional)  representational  framework  for 
planning  avoiding  classic  limitations 

-  Observed  2  to  90  times  speed  up  solving  classic  planning  problems  such  as 
Blocks  World  and  Eight-Square 

-  Glasgow  &  Malton  2003 

-  Applied  array  theory  to  capture  spatial  relations  as  symbolic  arrays 

-  Observed  computational  benefit  from  the  implicit  representation  of  spatial  and 
topological  constraints 
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-  “Conjoint  Analysis  Tutorial” 
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-  “Statistical  Design  and  Analysis  of  Experiments,  with  Applications  to 
Engineering  and  Science” 
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-  Wiley 

-  “Statistical  Design  and  Analysis  of  Experiments” 
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